Nagios plugin TLS mutual auth support

[bworrell]

Is TLS mutual authentication supported by the nagios integration plugin?

I currently run nagios 3.5.1 with SSL authentication under apache 2.4 with SSLVerifyClient require and I can't seem to get the nagiosActionExecutor to present my client certificate in the ssl handshake with apache when performing a nagios action.

I've installed v2.19.0 of opsgenie-nagios:

$ dpkg -s opsgenie-nagios
Package: opsgenie-nagios
Status: install ok installed
Priority: extra
Section: default
Installed-Size: 27997
Maintainer: OpsGenie
Architecture: all
Version: 2.19.0
<!-- snip -->

I am running marid with the following additional MARID_JVM options via /etc/opsgenie/profile:

SSL_OPTS="\
-Djavax.net.debug=all \
-Djavax.net.ssl.keyStoreType=pkcs12 \
-Djavax.net.ssl.keyStore=/path/to/keystore/opsgenie.p12 \
-Djavax.net.ssl.keyStorePassword=[removed for github] \
-Djavax.net.ssl.trustStore=/path/to/cacerts \
-Djavax.net.ssl.trustStoreType=jks \
-Djsse.enableSNIExtension=false"

export MARID_JVM="$MARID_JVM $SSL_OPTS"

The ssl debug logs don't show a client certificate being sent and I end up with a peer not authenticated exception message:

17/12/23 20:07:31.881 WARN: Exception occurred while executing script [nagiosActionExecutor.groovy]. Reason: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
org.codehaus.groovy.runtime.InvokerInvocationException: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:97)
        at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:233)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1054)
        at groovy.lang.ExpandoMetaClass.invokeMethod(ExpandoMetaClass.java:1110)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:884)
        at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:704)
        at groovy.lang.GroovyObjectSupport.invokeMethod(GroovyObjectSupport.java:44)
        at groovy.lang.Script.invokeMethod(Script.java:78)
        at com.ifountain.opsgenie.client.script.GroovyCompiledScriptEngine.doRun(GroovyCompiledScriptEngine.java:21)
        at com.ifountain.opsgenie.client.script.AbstractCompiledScriptEngine.runScript(AbstractCompiledScriptEngine.java:30)
        at com.ifountain.opsgenie.client.script.ScriptManager.runScript(ScriptManager.java:62)
        at com.ifountain.opsgenie.client.marid.alert.AlertActionUtils.executeActionScript(AlertActionUtils.java:43)
        at com.ifountain.opsgenie.client.marid.alert.PubnubAlertActionListener.processMessage(PubnubAlertActionListener.java:162)
        at com.ifountain.opsgenie.client.marid.alert.PubnubAlertActionListener$2$1$1.run(PubnubAlertActionListener.java:98)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
        at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:421)
        at org.apache.http.conn.ssl.AbstractVerifier.verify(AbstractVerifier.java:126)
        at org.apache.http.conn.ssl.SSLSocketFactory.connectSocket(SSLSocketFactory.java:437)
        at org.apache.http.impl.conn.DefaultClientConnectionOperator.openConnection(DefaultClientConnectionOperator.java:180)
        at org.apache.http.impl.conn.ManagedClientConnectionImpl.open(ManagedClientConnectionImpl.java:294)
        at org.apache.http.impl.client.DefaultRequestDirector.tryConnect(DefaultRequestDirector.java:643)
        at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:479)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1138)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1076)
        at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:1050)
        at com.ifountain.opsgenie.client.http.OpsGenieHttpClient$OpsgenieHttpClientRetryMechanism.execute(OpsGenieHttpClient.java:391)
        at com.ifountain.opsgenie.client.http.OpsGenieHttpClient.executeHttpMethod(OpsGenieHttpClient.java:248)
        at com.ifountain.opsgenie.client.http.OpsGenieHttpClient.post(OpsGenieHttpClient.java:108)
        at com.ifountain.opsgenie.client.http.OpsGenieHttpClient.post(OpsGenieHttpClient.java:90)
        at com.ifountain.opsgenie.client.http.OpsGenieHttpClient.post(OpsGenieHttpClient.java:78)
        at com.ifountain.opsgenie.client.http.OpsGenieHttpClient$post.call(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:42)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
        at nagiosActionExecutor.postToNagios(nagiosActionExecutor.groovy:130)
        at nagiosActionExecutor$postToNagios.callCurrent(Unknown Source)
        at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:46)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:133)
        at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:141)
        at nagiosActionExecutor.run(nagiosActionExecutor.groovy:79)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:90)
        ... 18 more

I can get other SSL code (like this) to connect to apache with the same jvm options.

It seems like this line might be the source of the issue, since it's passing in null for the key manager array. Or maybe it's the way the groovy scripts are invoked (e.g., not passing the jvm options)?

Sorry if this is posted to the wrong repo or is just a pebkac issue!

[celalemre]

Hi @bworrell !

Thank you very much for reaching us. We'll investigate the problem and once it's resolved, we'll notify you.

[celalemre]

Hi @bworrell,

We’ve tried to reproduce the problem and we successfully reproduced it by using Java 6 version. We tried both OpenJDK 8 and Oracle JDK 8 and we didn’t see this problem with these versions. Are you using Java 6? If so, upgrading your Java to version 7 or above will solve the problem.

[bworrell]

Hey @celalemre, sorry for the late reply. I am not 100% certain of everything I did during testing as it's been a while, but it looks like we are running everything on Java 1.7.

During my testing, I know I compiled a small SSL client test app on Java 1.7 and 1.8 to verify that I could get a java app handshaking with apache.

I then integrated the opsgenieclient SSLSocketFactory code into my client and verified it longer presented a client certificate in the handshake when using the same JVM options.

I can retest some of this and post my findings. Sorry for not leaving more details originally!