spring-boot tomcat redeploy permgen leak due to logging

Viewed 143

Ran into a memory leak issue with redeploying a spring boot 2.5 application on tomcat 9.

Debugging process:

  1. Tomcat startup.bat
  2. Deploy war
  3. Reload/undeploy war
  4. Run tomcat find leaks
  5. Check loaded classes in visual vm

Tomcat findleaks reports a leak and visual VM indicates that the classes aren't unloaded.

Next steps:

  1. Take a heapdump
  2. Load dump in Eclipse Memory Analyzer
  3. List objexts with incoming references and filter to org.apache.catalina.loader.ParallelWebappClassLoader

This also lists classloaders with a DESTROYED state. For a destroyed classloader do the following:

  1. Path to GC roots > Excluce all phantom/weak/soft/etc references

This indicates there may still be a strong reference relating to org.apache.logging.log4j.core.jmx.StatusLoggerAdminMBean

eclipse memory analyzer

Attempted to recreate the issue in a simple spring boot project using the logback logging and log4j2. Both show the same scenario but with a different cause (shutdown hooks).

Debugging and code based on the spring boot starter can be found here: https://github.com/webbres/spring-boot-permgen-issue/tree/master/debugging

At this stage I'm unsure if the permgen leak in my actual application is related to the one in the linked repo.

Are there additional steps that need to be taken that the spring boot starter isn't handling w.r.t to using the logging frameworks in a war? I have tried adding org.apache.logging.log4j:log4j-web as a dependency but this doesn't appear to resolve the shutdown hook issue in the demo project.

Feels like I'm missing something...

Adding -Dlog4j2.disable.jmx=true to the CATALINA_OPTS and logging.register-shutdown-hook=false to the application.properties I no longer get and references when filtering out phatom/weak/soft/etc references however I do still see the DESTROYED instance of the ParallelWebappClassLoader and the classes aren't being unloaded

0 Answers
Related