Since switching from Tomcat CP (spring boot 1 default) to Hikari (spring boot 2 default) we've started seeing many instances of:
EXCEPTION STACK TRACE:
** BEGIN NESTED EXCEPTION **
javax.net.ssl.SSLException
MESSAGE: closing inbound before receiving peer's close_notify
STACKTRACE:
javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:129)
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:308)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:264)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:255)
at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:645)
at java.base/sun.security.ssl.SSLSocketImpl.shutdownInput(SSLSocketImpl.java:624)
at com.mysql.cj.protocol.a.NativeProtocol.quit(NativeProtocol.java:1312)
at com.mysql.cj.NativeSession.quit(NativeSession.java:182)
at com.mysql.cj.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:1750)
at com.mysql.cj.jdbc.ConnectionImpl.close(ConnectionImpl.java:720)
at com.zaxxer.hikari.pool.PoolBase.quietlyCloseConnection(PoolBase.java:135)
at com.zaxxer.hikari.pool.HikariPool.lambda$closeConnection$1(HikariPool.java:441)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Environment
- Spring Boot 2.1.1.RELEASE
- Java 11
- mysql-connector-java 8.0.13
- HikariCP 3.2.0
Database:
- RDS Aurora MySql 5.7.12 (default param group)
Configuration (Spring Boot)
Settings:
spring.datasource.hikari.transactionIsolation=TRANSACTION_REPEATABLE_READ
spring.datasource.hikari.minimumIdle=10
spring.datasource.hikari.idleTimeout=300000
spring.datasource.hikari.maximumPoolSize=20
spring.datasource.hikari.connectionTimeout=5000
spring.datasource.hikari.maxLifetime=900000
spring.datasource.hikari.validationTimeout=1000
Is there a setting which I'm missing, perhaps my idle times should be set much lower?
We have not (yet) experienced any obvious bad side effects of this, i.e. the application appears to continue running without issue, but this stacktrace appears frequently (perhaps every 4 seconds.
Database Settings
If I connect to mysql via the cli and run show variables; and grep for timeout related values, I see:
| connect_timeout | 10 |
| delayed_insert_timeout | 300 |
| have_statement_timeout | YES |
| innodb_flush_log_at_timeout | 1 |
| innodb_lock_wait_timeout | 50 |
| innodb_rollback_on_timeout | OFF |
| interactive_timeout | 28800 |
| lock_wait_timeout | 31536000 |
| net_read_timeout | 30 |
| net_write_timeout | 60 |
| rpl_stop_slave_timeout | 31536000 |
| slave_net_timeout | 60 |
| wait_timeout | 28800 |