Multithreading java application is failing sometimes to get all threads on Websphere

Viewed 484

I have an exhaustive java application (JRE 1.8.0_201) that requires to be executed in threads, It is deployed on IBM Websphere 9.

At WAS startup time the application executions seems to be fine, however sometimes later the multithreading capability is lost and the executions occurs using only one thread.

If we restart again the websphere the situation turns into normality, the executions takes again the proper parallel threads.

Is there some WAS configurations that I'm missing? or at java or OS side?

Below the HW architecture on Linux:

  • Architecture: x86_64
  • CPU op-mode (s): 32-bit, 64-bit
  • Byte Order: Little Endian
  • CPU (s): 4
  • On-line CPU (s) list: 0-3
  • Thread (s) per core: 1
  • Core (s) per socket: 1
  • Socket (s): 4
2 Answers

To get maximum information about the problem:

Thread Dump

Creates a listing of the current state of all threads within JVM. It’s advisable to do this listing repeatedly (in differently named outputs, don’t overwrite your dumps) within a few tens of seconds/minutes and in your case maybe hours. This allows you to capture changes and possible dead-lock. If nothing has changed and thread is on the same row, then there might be a problem.

How to make thread dump:

Open the console (CMD), switch to “/Java/jdkx.x.x_xx /bin” (JDK) using the “cd” command. Then type “jstack JVM_PID > /thread_dump_1“. Replace the “JVM_PID” with the process ID of running JVM (Task Manager – Details – PID column).

You can also use tool like VisualVM to take a thread dump:

enter image description here

How read thread dumps:

Great article: https://dzone.com/articles/how-to-read-a-thread-dump

Look for blocked and waiting threads. Probably there will be multiple threads stucked on same line and only one thread making some progress.

How read thread dumps in your situation:

I would make a few thread dumps right after the restart and then when problem occurs. If you can distinguish when one thread ends and another one get time for progress I would make another thread dump and compare them (before first end and after second start working).

Websphere has a thread pool for each server.

My understanding from your query is that you have an multithreading program where there is no restriction on the number of threads created.

So, in such a scenario there would be a point where websphere would run out of all the threads in the pool and then would wait for previous threads to close and would assign the new thread to your program as and when a new thread is available in its thread pool.

My first recommendation would be to user Java multithreading framework like Executor pool. I would recommend that you have a limit on the number of threads to be created.

ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(10);
//10 is the max number of threads for this threadpool
executor.execute(<your Runnable Object here>);

But, if you still do not want to restrict then you can use:

ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newCachedThreadPool();
//This will create thread as and when needed however, this would have a better management of threads as compared to simply creating threads.
executor.execute(<your Runnable Object here>);

Please note that still you may run out of threads and memory as there is no limitation.

In Webspehere you can set the number of threads in : Servers --> All Servers --> --> Additional Properties --> Thread pools --> WebContainer Minimum Size, Maximum Size

Related