Java 1.8.0_271 non-compliant to TLS RFC: during handshake, server closes TCP connection without close_notify or alert

user_14777035 0
Viewed 727

We used to Weblogic 12c servers on old java versions (1.8.0_51). After upgrading to 1.8.0_271, we have an intermittent issue: after working fine for a few minutes, TLS handshakes incoming to Weblogic fail.

The observed behaviour is as follows:

Client --[TCP SYN]--> Weblogic
Weblogic --[TCP SYN, ACK]--> Client
Client --[TCP ACK]--> Weblogic
Client --[Client Hello]--> Weblogic
Weblogic --[TCP FIN, ACK]--> Client

The handshake is being made using TLS 1.3.

This seem very unusual. I read through RFC 8446 (The Transport Layer Security (TLS) Protocol Version 1.3), and I was not able to find any reason why the server would close the TCP connection without sending an alert or a close_notify first. I'd even say that, from my understanding, the JVM behaviour is non-compliant:

   Each party MUST send a "close_notify" alert before closing its write
   side of the connection, unless it has already sent some error alert.

Going back to Java 1.8.0_51 is fixing the issue.

We have this issue on both Linux and Solaris servers, and we see this issue when doing tests both remotely and locally (on the CLI of the server, using openssl s_client). So there is no way that a firewall, WAF, IPS or anything else could cause this issue.

tcpdump wireshark openssl ssl java
2 Answers

As a workaround, you could try disabling TLSv1.3 on your Java 1.8.0_271 version.

Even if there are more consistent ways to do so, in order to test it in a fast way, you could define it dynamically; Place this sentence directly in your code, before the connection related methods:

java.lang.System.setProperty("jdk.tls.client.protocols", "TLSv1,TLSv1.1,TLSv1.2");

Another option would be definning it statically. For example, while launching it from a terminal:

$ java -Djdk.tls.client.protocols="TLSv1,TLSv1.1,TLSv1.2" YourJavaProgram

This will disable any protocol not included in the list, so no TLSV1.3 will ever be accepted by your client; The server will now ask for the next protocol in its list, by default TLSv1.2. This is properly enabled on your side and seems to work without issues on your environment.

I'f im not wrong, 1.8.0_271 changed some specifications regarding TLS3, such as maximum number of certs allowed, etc. The core issue could be related to this specifications. This workaround is more a temporal fix than a solution, but still, hope it helps somehow.

user_21489530

I have think I have finally found the cause of this issue: there is this JAR that is found in the CLASSPATH: bcmail-jdk14-1.38.jar. It is from 2008, and has been built for JDK 1.4. It contains an org.bouncycastle.jce.provider class.

When we enable Weblogic's SSL debugging, we can see that it is the cause - and this is an issue that is known to happen when using outdated Bouncy Castle libraries.

Now, there's still one thing I'm not quite sure about: when we start Weblogic, it is doing fine for 5-40 minutes. It is able to process hundreds of client requests and TLS handshake. And at one point, it just stops working: all TLS handshakes that would process fine before are now receiving TCP FIN. It's almost like if the bouncycastle JCE provider overrides the default JCE provider on runtime, after a long delay. I'm not sure if that's possible.

####<2020-12-07 15 h 53> <Debug> <SecuritySSL> <hostname.domain.com> <AdminServer> <ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'> <<WLS Kernel>> <> <5226be2e-62b3-4804-8063-b75cf217d5f2-00000068> <1607374405613> <[severity-value: 128] [rid: 0] [partition-id: 0] [partition-name: DOMAIN] > <BEA-000000> <[Thread[ExecuteThread: '1' for queue: 'weblogic.socket.Muxer',5,Thread Group for Queue: 'weblogic.socket.Muxer']]weblogic.security.SSL.jsseadapter: SSLENGINE: Exception occurred during SSLEngine.unwrap(ByteBuffer,ByteBuffer[]).
java.lang.RuntimeException: Could not generate ECDH keypair
    at sun.security.ssl.ECDHKeyExchange$ECDHEPossession.<init>(ECDHKeyExchange.java:116)
    at sun.security.ssl.ECDHKeyExchange$ECDHEPossessionGenerator.createPossession(ECDHKeyExchange.java:230)
    at sun.security.ssl.SSLKeyExchange$T12KeyAgreement.createPossession(SSLKeyExchange.java:377)
    at sun.security.ssl.SSLKeyExchange.createPossessions(SSLKeyExchange.java:88)
    at sun.security.ssl.ServerHello$T12ServerHelloProducer.chooseCipherSuite(ServerHello.java:432)
    at sun.security.ssl.ServerHello$T12ServerHelloProducer.produce(ServerHello.java:294)
    at sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:420)
    at sun.security.ssl.ClientHello$T12ClientHelloConsumer.consume(ClientHello.java:964)
    at sun.security.ssl.ClientHello$ClientHelloConsumer.onClientHello(ClientHello.java:718)
    at sun.security.ssl.ClientHello$ClientHelloConsumer.consume(ClientHello.java:683)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:376)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:983)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:970)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:917)
    at weblogic.socket.JSSEFilterImpl.doTasks(JSSEFilterImpl.java:211)
    at weblogic.socket.JSSEFilterImpl.doHandshake(JSSEFilterImpl.java:113)
    at weblogic.socket.JSSEFilterImpl.isMessageComplete(JSSEFilterImpl.java:308)
    at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:975)
    at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:916)
    at weblogic.socket.NIOSocketMuxer.process(NIOSocketMuxer.java:596)
    at weblogic.socket.NIOSocketMuxer.processSockets(NIOSocketMuxer.java:560)
    at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:30)
    at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:43)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
Caused By: java.security.InvalidAlgorithmParameterException: parameter object not a ECParameterSpec
    at org.bouncycastle.jce.provider.JDKKeyPairGenerator$EC.initialize(Unknown Source)
    at sun.security.ssl.ECDHKeyExchange$ECDHEPossession.<init>(ECDHKeyExchange.java:111)
    at sun.security.ssl.ECDHKeyExchange$ECDHEPossessionGenerator.createPossession(ECDHKeyExchange.java:230)
    at sun.security.ssl.SSLKeyExchange$T12KeyAgreement.createPossession(SSLKeyExchange.java:377)
    at sun.security.ssl.SSLKeyExchange.createPossessions(SSLKeyExchange.java:88)
    at sun.security.ssl.ServerHello$T12ServerHelloProducer.chooseCipherSuite(ServerHello.java:432)
    at sun.security.ssl.ServerHello$T12ServerHelloProducer.produce(ServerHello.java:294)
    at sun.security.ssl.SSLHandshake.produce(SSLHandshake.java:420)
    at sun.security.ssl.ClientHello$T12ClientHelloConsumer.consume(ClientHello.java:964)
    at sun.security.ssl.ClientHello$ClientHelloConsumer.onClientHello(ClientHello.java:718)
    at sun.security.ssl.ClientHello$ClientHelloConsumer.consume(ClientHello.java:683)
    at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:376)
    at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:983)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:970)
    at java.security.AccessController.doPrivileged(Native Method)
    at sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:917)
    at weblogic.socket.JSSEFilterImpl.doTasks(JSSEFilterImpl.java:211)
    at weblogic.socket.JSSEFilterImpl.doHandshake(JSSEFilterImpl.java:113)
    at weblogic.socket.JSSEFilterImpl.isMessageComplete(JSSEFilterImpl.java:308)
    at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:975)
    at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:916)
    at weblogic.socket.NIOSocketMuxer.process(NIOSocketMuxer.java:596)
    at weblogic.socket.NIOSocketMuxer.processSockets(NIOSocketMuxer.java:560)
    at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:30)
    at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:43)
    at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:147)
    at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:119)
user_147770350
Related