I have the following code snippet in my actual code. I have mutiple consumers listening to a queue.
@RabbitListener
private void abc(ETLConfigDTO config){
try{
log.info("load started");
loadService.loadData(config);
}
catch(Exception e){
log.error("Load failed"):
}
finally{
log.info("finished processing"):
}
}
loadData() takes few minutes to few hours of processing. Its kind of etl processing. There is intensove logging inside this method, so i know in which state the process is. The problem is that the process is kind of stuck inside loadPlans() method. The message in the queue is in unacknowledged state since it is still processing which i need in that way. There is no exception since catch is not printing anything or even the finally block. I also have a spring cron (5 minutes interval) in the same class which is also running fine and doing its tasks.
The point to note is that this is running fine if I not use rabbit amqp.
Is there any connection/network drop? Or any timeout? Or the main thread is hung/dead? I am really not understanding what is happening here.
Thanks in advance.
UPDATE:
Thanks Gary,
I see this in jstack 19:
"SimpleAsyncTaskExecutor-1" #25 prio=5 os_prio=0 tid=0x00007f5615b3d800 nid=0x2f runnable [0x00007f56703cd000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:465)
at sun.security.ssl.InputRecord.read(InputRecord.java:503)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
- locked <0x000000067a6bada8> (a java.lang.Object)
at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940)
at sun.security.ssl.AppInputStream.read(AppInputStream.java:105)
- locked <0x000000067a6baea0> (a sun.security.ssl.AppInputStream)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
- locked <0x000000067a717cb8> (a java.io.BufferedInputStream)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:735)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
- locked <0x000000067a6b4b60> (a sun.net.www.protocol.https.DelegateHttpsURLConnection)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
- locked <0x000000067a6b4b60> (a sun.net.www.protocol.https.DelegateHttpsURLConnection)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:338)
at org.springframework.http.client.SimpleClientHttpResponse.getRawStatusCode(SimpleClientHttpResponse.java:48)
at org.springframework.http.client.AbstractClientHttpResponse.getStatusCode(AbstractClientHttpResponse.java:33)
at org.springframework.web.client.DefaultResponseErrorHandler.getHttpStatusCode(DefaultResponseErrorHandler.java:56)
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:50)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:602)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:570)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:530)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:448)
..
...
...
...
Please advise.
NEW UPDATE: I have increase memory -XX:MaxMetaspaceSize=1024M -Xms4096M -Xmx4096M
The thread is being stuck on oracle connection now.
"SimpleAsyncTaskExecutor-1" #25 prio=5 os_prio=0 tid=0x00007ff6102c8800 nid=0x33 runnable [0x00007ff619ad9000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at oracle.net.ns.Packet.receive(Packet.java:300)
at oracle.net.ns.DataPacket.receive(DataPacket.java:106)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:260)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:185)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:102)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:124)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:80)
at oracle.jdbc.driver.T4CMAREngine.unmarshalUB1(T4CMAREngine.java:1137)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:290)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CStatement.doOall8(T4CStatement.java:193)
at oracle.jdbc.driver.T4CStatement.executeForRows(T4CStatement.java:1033)
at oracle.jdbc.driver.OracleStatement.executeBatch(OracleStatement.java:4536)
- locked <0x00000007b01c6b20> (a oracle.jdbc.driver.T4CConnection)
at oracle.jdbc.driver.OracleStatementWrapper.executeBatch(OracleStatementWrapper.java:230)
at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:572)
at org.springframework.jdbc.core.JdbcTemplate$1BatchUpdateStatementCallback.doInStatement(JdbcTemplate.java:559)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:405)
at org.springframework.jdbc.core.JdbcTemplate.batchUpdate(JdbcTemplate.java:611)
...
...