We have a Spring Batch Application running in AKS environment. We are facing com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed. intermittently.
[144] ERROR : 03.10.2022:0454 (54.161) [[]main] AbstractJob: Encountered fatal error executing job org.springframework.batch.core.JobExecutionException: Flow execution ended unexpectedly
at org.springframework.batch.core.job.flow.FlowJob.doExecute(FlowJob.java:141) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.AbstractJob.execute(AbstractJob.java:281) [irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.launch.support.SimpleJobLauncher$1.run(SimpleJobLauncher.java:120) [irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.core.task.SyncTaskExecutor.execute(SyncTaskExecutor.java:49) [irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:114) [irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.launch.support.CommandLineJobRunner.start(CommandLineJobRunner.java:349) [irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.launch.support.CommandLineJobRunner.main(CommandLineJobRunner.java:574) [irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?] Caused by: org.springframework.batch.core.job.flow.FlowExecutionException: Ended flow=multithreadedItemSkuProcessing.7 at state=multithreadedItemSkuProcessing.7.applyItem8 with exception
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:152) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_292]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] Caused by: org.springframework.transaction.TransactionSystemException: Could not roll back JDBC transaction; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:288) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:846) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:823) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:493) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:264) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at com.sun.proxy.$Proxy18.getLastStepExecution(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:106) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_292]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292] Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:234) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.checkClosed(SQLServerConnection.java:1090) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.rollback(SQLServerConnection.java:3195) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.apache.commons.dbcp.DelegatingConnection.rollback(DelegatingConnection.java:368) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.apache.commons.dbcp.DelegatingConnection.rollback(DelegatingConnection.java:368) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.rollback(PoolingDataSource.java:323) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.jdbc.datasource.DataSourceTransactionManager.doRollback(DataSourceTransactionManager.java:285) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processRollback(AbstractPlatformTransactionManager.java:846) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.rollback(AbstractPlatformTransactionManager.java:823) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.completeTransactionAfterThrowing(TransactionAspectSupport.java:493) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:264) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:94) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at com.sun.proxy.$Proxy18.getLastStepExecution(Unknown Source) ~[?:?]
at org.springframework.batch.core.job.SimpleStepHandler.handleStep(SimpleStepHandler.java:106) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.JobFlowExecutor.executeStep(JobFlowExecutor.java:61) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.StepState.handle(StepState.java:60) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.resume(SimpleFlow.java:144) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.SimpleFlow.start(SimpleFlow.java:124) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:91) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at org.springframework.batch.core.job.flow.support.state.SplitState$1.call(SplitState.java:90) ~[irol-bat-applybatch-jobs-2.2.12-SNAPSHOT.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_292]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_292]
We have implemented the retry logic as well in the job definition, but retry logic is also not working. It looks like it is not picking the mentioned com.microsoft.sqlserver.jdbc.SQLServerException as it is not the parent exception.
The application uses Azure SQL Server DataBase and Apache DBCP Connection Pooling with the following configuration
So how can we get rid of this issue or how can we make retryable logic work?