How to load JAR files dynamically at Runtime?

Viewed 357383

Why is it so hard to do this in Java? If you want to have any kind of module system you need to be able to load JAR files dynamically. I'm told there's a way of doing it by writing your own ClassLoader, but that's a lot of work for something that should (in my mind at least) be as easy as calling a method with a JAR file as its argument.

Any suggestions for simple code that does this?

20 Answers

The reason it's hard is security. Classloaders are meant to be immutable; you shouldn't be able to willy-nilly add classes to it at runtime. I'm actually very surprised that works with the system classloader. Here's how you do it making your own child classloader:

URLClassLoader child = new URLClassLoader(
        new URL[] {myJar.toURI().toURL()},
        this.getClass().getClassLoader()
);
Class classToLoad = Class.forName("com.MyClass", true, child);
Method method = classToLoad.getDeclaredMethod("myMethod");
Object instance = classToLoad.newInstance();
Object result = method.invoke(instance);

Painful, but there it is.

The following solution is hackish, as it uses reflection to bypass encapsulation, but it works flawlessly:

File file = ...
URL url = file.toURI().toURL();

URLClassLoader classLoader = (URLClassLoader)ClassLoader.getSystemClassLoader();
Method method = URLClassLoader.class.getDeclaredMethod("addURL", URL.class);
method.setAccessible(true);
method.invoke(classLoader, url);

You should take a look at OSGi, e.g. implemented in the Eclipse Platform. It does exactly that. You can install, uninstall, start and stop so called bundles, which are effectively JAR files. But it does a little more, as it offers e.g. services that can be dynamically discovered in JAR files at runtime.

Or see the specification for the Java Module System.

While most solutions listed here are either hacks (pre JDK 9) hard to configure (agents) or just don't work anymore (post JDK 9) I find it really surprising that nobody mentioned a clearly documented method.

You can create a custom system class loader and then you're free to do whatever you wish. No reflection required and all classes share the same classloader.

When starting the JVM add this flag:

java -Djava.system.class.loader=com.example.MyCustomClassLoader

The classloader must have a constructor accepting a classloader, which must be set as its parent. The constructor will be called on JVM startup and the real system classloader will be passed, the main class will be loaded by the custom loader.

To add jars just call ClassLoader.getSystemClassLoader() and cast it to your class.

Check out this implementation for a carefully crafted classloader. Please note, you can change the add() method to public.

With Java 9, the answers with URLClassLoader now give an error like:

java.lang.ClassCastException: java.base/jdk.internal.loader.ClassLoaders$AppClassLoader cannot be cast to java.base/java.net.URLClassLoader

This is because the class loaders used have changed. Instead, to add to the system class loader, you can use the Instrumentation API through an agent.

Create an agent class:

package ClassPathAgent;

import java.io.IOException;
import java.lang.instrument.Instrumentation;
import java.util.jar.JarFile;

public class ClassPathAgent {
    public static void agentmain(String args, Instrumentation instrumentation) throws IOException {
        instrumentation.appendToSystemClassLoaderSearch(new JarFile(args));
    }
}

Add META-INF/MANIFEST.MF and put it in a JAR file with the agent class:

Manifest-Version: 1.0
Agent-Class: ClassPathAgent.ClassPathAgent

Run the agent:

This uses the byte-buddy-agent library to add the agent to the running JVM:

import java.io.File;

import net.bytebuddy.agent.ByteBuddyAgent;

public class ClassPathUtil {
    private static File AGENT_JAR = new File("/path/to/agent.jar");

    public static void addJarToClassPath(File jarFile) {
        ByteBuddyAgent.attach(AGENT_JAR, String.valueOf(ProcessHandle.current().pid()), jarFile.getPath());
    }
}

Here is a quick workaround for Allain's method to make it compatible with newer versions of Java:

ClassLoader classLoader = ClassLoader.getSystemClassLoader();
try {
    Method method = classLoader.getClass().getDeclaredMethod("addURL", URL.class);
    method.setAccessible(true);
    method.invoke(classLoader, new File(jarPath).toURI().toURL());
} catch (NoSuchMethodException e) {
    Method method = classLoader.getClass()
            .getDeclaredMethod("appendToClassPathForInstrumentation", String.class);
    method.setAccessible(true);
    method.invoke(classLoader, jarPath);
}

Note that it relies on knowledge of internal implementation of specific JVM, so it's not ideal and it's not a universal solution. But it's a quick and easy workaround if you know that you are going to use standard OpenJDK or Oracle JVM. It might also break at some point in future when new JVM version is released, so you need to keep that in mind.

The best I've found is org.apache.xbean.classloader.JarFileClassLoader which is part of the XBean project.

Here's a short method I've used in the past, to create a class loader from all the lib files in a specific directory

public void initialize(String libDir) throws Exception {
    File dependencyDirectory = new File(libDir);
    File[] files = dependencyDirectory.listFiles();
    ArrayList<URL> urls = new ArrayList<URL>();
    for (int i = 0; i < files.length; i++) {
        if (files[i].getName().endsWith(".jar")) {
        urls.add(files[i].toURL());
        //urls.add(files[i].toURI().toURL());
        }
    }
    classLoader = new JarFileClassLoader("Scheduler CL" + System.currentTimeMillis(), 
        urls.toArray(new URL[urls.size()]), 
        GFClassLoader.class.getClassLoader());
}

Then to use the classloader, just do:

classLoader.loadClass(name);

I know I'm late to the party, but I have been using pf4j, which is a plug-in framework, and it works pretty well.

PF4J is a microframework and the aim is to keep the core simple but extensible.

An example of plugin usage:

Define an extension point in your application/plugin using ExtensionPoint interface marker:

public interface Greeting extends ExtensionPoint {

    String getGreeting();

}

Create an extension using @Extension annotation:

@Extension
public class WelcomeGreeting implements Greeting {

    public String getGreeting() {
        return "Welcome";
    }

}

Then you can load and unload the plugin as you wish:

public static void main(String[] args) {

    // create the plugin manager
    PluginManager pluginManager = new JarPluginManager(); // or "new ZipPluginManager() / new DefaultPluginManager()"

    // start and load all plugins of application
    pluginManager.loadPlugins();
    pluginManager.startPlugins();

    // retrieve all extensions for "Greeting" extension point
    List<Greeting> greetings = pluginManager.getExtensions(Greeting.class);
    for (Greeting greeting : greetings) {
        System.out.println(">>> " + greeting.getGreeting());
    }

    // stop and unload all plugins
    pluginManager.stopPlugins();
    pluginManager.unloadPlugins();

}

For further details please refer to the documentation

Another working solution using Instrumentation that works for me. It has the advantage of modifying the class loader search, avoiding problems on class visibility for dependent classes:

Create an Agent Class

For this example, it has to be on the same jar invoked by the command line:

package agent;

import java.io.IOException;
import java.lang.instrument.Instrumentation;
import java.util.jar.JarFile;

public class Agent {
   public static Instrumentation instrumentation;

   public static void premain(String args, Instrumentation instrumentation) {
      Agent.instrumentation = instrumentation;
   }

   public static void agentmain(String args, Instrumentation instrumentation) {
      Agent.instrumentation = instrumentation;
   }

   public static void appendJarFile(JarFile file) throws IOException {
      if (instrumentation != null) {
         instrumentation.appendToSystemClassLoaderSearch(file);
      }
   }
}

Modify the MANIFEST.MF

Adding the reference to the agent:

Launcher-Agent-Class: agent.Agent
Agent-Class: agent.Agent
Premain-Class: agent.Agent

I actually use Netbeans, so this post helps on how to change the manifest.mf

Running

The Launcher-Agent-Class is only supported on JDK 9+ and is responsible for loading the agent without explicitly defining it on the command line:

 java -jar <your jar>

The way that works on JDK 6+ is defining the -javaagent argument:

java -javaagent:<your jar> -jar <your jar>

Adding new Jar at Runtime

You can then add jar as necessary using the following command:

Agent.appendJarFile(new JarFile(<your file>));

I did not find any problems using this on documentation.

Another version of the hackish solution from Allain, that also works on JDK 11:

File file = ...
URL url = file.toURI().toURL();
URLClassLoader sysLoader = new URLClassLoader(new URL[0]);

Method sysMethod = URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{URL.class});
sysMethod.setAccessible(true);
sysMethod.invoke(sysLoader, new Object[]{url});

On JDK 11 it gives some deprecation warnings but serves as a temporary solution those who use Allain solution on JDK 11.

In case anyone searches for this in the future, this way works for me with OpenJDK 13.0.2.

I have many classes that I need to instantiate dynamically at runtime, each potentially with a different classpath.

In this code, I already have an object called pack, that holds some metadata about the class I am trying to load. The getObjectFile() method returns the location of the class file for the class. The getObjectRootPath() method returns the path to the bin/ directory containing the class files that include the class I am trying to instantiate. The getLibPath() method returns the path to a directory containing the jar files constituting the classpath for the module the class is a part of.

File object = new File(pack.getObjectFile()).getAbsoluteFile();
Object packObject;
try {
    URLClassLoader classloader;

    List<URL> classpath = new ArrayList<>();
    classpath.add(new File(pack.getObjectRootPath()).toURI().toURL());
    for (File jar : FileUtils.listFiles(new File(pack.getLibPath()), new String[] {"jar"}, true)) {
        classpath.add(jar.toURI().toURL());
    }
    classloader = new URLClassLoader(classpath.toArray(new URL[] {}));

    Class<?> clazz = classloader.loadClass(object.getName());
    packObject = clazz.getDeclaredConstructor().newInstance();

} catch (Exception e) {
    e.printStackTrace();
    throw e;
}
return packObject;

I was using the Maven dependency: org.xeustechnologies:jcl-core:2.8 to do this before, but after moving past JDK 1.8, it sometimes froze and never returned being stuck "waiting for references" at Reference::waitForReferencePendingList().

I am also keeping a map of class loaders so that they can be reused if the class I am trying to instantiate is in the same module as a class that I have already instantiated, which I would recommend.

please take a look at this project that i started: proxy-object lib

This lib will load jar from file system or any other location. It will dedicate a class loader for the jar to make sure there are no library conflicts. Users will be able to create any object from the loaded jar and call any method on it. This lib was designed to load jars compiled in Java 8 from the code base that supports Java 7.

To create an object:

    File libDir = new File("path/to/jar");

    ProxyCallerInterface caller = ObjectBuilder.builder()
            .setClassName("net.proxy.lib.test.LibClass")
            .setArtifact(DirArtifact.builder()
                    .withClazz(ObjectBuilderTest.class)
                    .withVersionInfo(newVersionInfo(libDir))
                    .build())
            .build();
    String version = caller.call("getLibVersion").asString();

ObjectBuilder supports factory methods, calling static functions, and call back interface implementations. i will be posting more examples on the readme page.

This can be a late response, I can do it as this (a simple example for fastutil-8.2.2.jar) using jhplot.Web class from DataMelt (http://jwork.org/dmelt)

import jhplot.Web;
Web.load("http://central.maven.org/maven2/it/unimi/dsi/fastutil/8.2.2/fastutil-8.2.2.jar"); // now you can start using this library

According to the documentation, this file will be download inside "lib/user" and then dynamically loaded, so you can start immediately using classes from this jar file in the same program.

I needed to load a jar file at runtime for both java 8 and java 9+ (above comments don't work for both of these versions). Here is the method to do it (using Spring Boot 1.5.2 if it may relate).

public static synchronized void loadLibrary(java.io.File jar) {
    try {            
        java.net.URL url = jar.toURI().toURL();
        java.lang.reflect.Method method = java.net.URLClassLoader.class.getDeclaredMethod("addURL", new Class[]{java.net.URL.class});
        method.setAccessible(true); /*promote the method to public access*/
        method.invoke(Thread.currentThread().getContextClassLoader(), new Object[]{url});
    } catch (Exception ex) {
        throw new RuntimeException("Cannot load library from jar file '" + jar.getAbsolutePath() + "'. Reason: " + ex.getMessage());
    }
}

For dynamic uploading of jar files, you can use my modification of URLClassLoader. This modification has no problem with changing the jar file during application operation, like the standard URLClassloader. All loaded jar files are loaded into RAM and thus independent of the original file.

In-memory jar and JDBC class loader

Related