Get a list of resources from classpath directory

Viewed 316105

I am looking for a way to get a list of all resource names from a given classpath directory, something like a method List<String> getResourceNames (String directoryName).

For example, given a classpath directory x/y/z containing files a.html, b.html, c.html and a subdirectory d, getResourceNames("x/y/z") should return a List<String> containing the following strings:['a.html', 'b.html', 'c.html', 'd'].

It should work both for resources in filesystem and jars.

I know that I can write a quick snippet with Files, JarFiles and URLs, but I do not want to reinvent the wheel. My question is, given existing publicly available libraries, what is the quickest way to implement getResourceNames? Spring and Apache Commons stacks are both feasible.

15 Answers

If you use apache commonsIO you can use for the filesystem (optionally with extension filter):

Collection<File> files = FileUtils.listFiles(new File("directory/"), null, false);

and for resources/classpath:

List<String> files = IOUtils.readLines(MyClass.class.getClassLoader().getResourceAsStream("directory/"), Charsets.UTF_8);

If you don't know if "directoy/" is in the filesystem or in resources you may add a

if (new File("directory/").isDirectory())

or

if (MyClass.class.getClassLoader().getResource("directory/") != null)

before the calls and use both in combination...

The Spring framework's PathMatchingResourcePatternResolver is really awesome for these things:

private Resource[] getXMLResources() throws IOException
{
    ClassLoader classLoader = MethodHandles.lookup().getClass().getClassLoader();
    PathMatchingResourcePatternResolver resolver = new PathMatchingResourcePatternResolver(classLoader);

    return resolver.getResources("classpath:x/y/z/*.xml");
}

Maven dependency:

<dependency>
    <groupId>org.springframework</groupId>
    <artifactId>spring-core</artifactId>
    <version>LATEST</version>
</dependency>

This should work (if spring is not an option):

public static List<String> getFilenamesForDirnameFromCP(String directoryName) throws URISyntaxException, UnsupportedEncodingException, IOException {
    List<String> filenames = new ArrayList<>();

    URL url = Thread.currentThread().getContextClassLoader().getResource(directoryName);
    if (url != null) {
        if (url.getProtocol().equals("file")) {
            File file = Paths.get(url.toURI()).toFile();
            if (file != null) {
                File[] files = file.listFiles();
                if (files != null) {
                    for (File filename : files) {
                        filenames.add(filename.toString());
                    }
                }
            }
        } else if (url.getProtocol().equals("jar")) {
            String dirname = directoryName + "/";
            String path = url.getPath();
            String jarPath = path.substring(5, path.indexOf("!"));
            try (JarFile jar = new JarFile(URLDecoder.decode(jarPath, StandardCharsets.UTF_8.name()))) {
                Enumeration<JarEntry> entries = jar.entries();
                while (entries.hasMoreElements()) {
                    JarEntry entry = entries.nextElement();
                    String name = entry.getName();
                    if (name.startsWith(dirname) && !dirname.equals(name)) {
                        URL resource = Thread.currentThread().getContextClassLoader().getResource(name);
                        filenames.add(resource.toString());
                    }
                }
            }
        }
    }
    return filenames;
}

My way, no Spring, used during a unit test:

URI uri = TestClass.class.getResource("/resources").toURI();
Path myPath = Paths.get(uri);
Stream<Path> walk = Files.walk(myPath, 1);
for (Iterator<Path> it = walk.iterator(); it.hasNext(); ) {
    Path filename = it.next();   
    System.out.println(filename);
}

With Spring it's easy. Be it a file, or folder, or even multiple files, there are chances, you can do it via injection.

This example demonstrates the injection of multiple files located in x/y/z folder.

import org.springframework.beans.factory.annotation.Value;
import org.springframework.core.io.Resource;
import org.springframework.stereotype.Service;

@Service
public class StackoverflowService {
    @Value("classpath:x/y/z/*")
    private Resource[] resources;

    public List<String> getResourceNames() {
        return Arrays.stream(resources)
                .map(Resource::getFilename)
                .collect(Collectors.toList());
    }
}

It does work for resources in the filesystem as well as in JARs.

I think you can leverage the [Zip File System Provider][1] to achieve this. When using FileSystems.newFileSystem it looks like you can treat the objects in that ZIP as a "regular" file.

In the linked documentation above:

Specify the configuration options for the zip file system in the java.util.Map object passed to the FileSystems.newFileSystem method. See the [Zip File System Properties][2] topic for information about the provider-specific configuration properties for the zip file system.

Once you have an instance of a zip file system, you can invoke the methods of the [java.nio.file.FileSystem][3] and [java.nio.file.Path][4] classes to perform operations such as copying, moving, and renaming files, as well as modifying file attributes.

The documentation for the jdk.zipfs module in [Java 11 states][5]:

The zip file system provider treats a zip or JAR file as a file system and provides the ability to manipulate the contents of the file. The zip file system provider can be created by [FileSystems.newFileSystem][6] if installed.

Here is a contrived example I did using your example resources. Note that a .zip is a .jar, but you could adapt your code to instead use classpath resources:

Setup

cd /tmp
mkdir -p x/y/z
touch x/y/z/{a,b,c}.html
echo 'hello world' > x/y/z/d
zip -r example.zip x

Java

import java.io.IOException;
import java.net.URI;
import java.nio.file.FileSystem;
import java.nio.file.FileSystems;
import java.nio.file.Files;
import java.util.Collections;
import java.util.stream.Collectors;

public class MkobitZipRead {

  public static void main(String[] args) throws IOException {
    final URI uri = URI.create("jar:file:/tmp/example.zip");
    try (
        final FileSystem zipfs = FileSystems.newFileSystem(uri, Collections.emptyMap());
    ) {
      Files.walk(zipfs.getPath("/")).forEach(path -> System.out.println("Files in zip:" + path));
      System.out.println("-----");
      final String manifest = Files.readAllLines(
          zipfs.getPath("x", "y", "z").resolve("d")
      ).stream().collect(Collectors.joining(System.lineSeparator()));
      System.out.println(manifest);
    }
  }

}

Output

Files in zip:/
Files in zip:/x/
Files in zip:/x/y/
Files in zip:/x/y/z/
Files in zip:/x/y/z/c.html
Files in zip:/x/y/z/b.html
Files in zip:/x/y/z/a.html
Files in zip:/x/y/z/d
-----
hello world

Neither of answers worked for me even though I had my resources put in resources folders and followed the above answers. What did make a trick was:

@Value("file:*/**/resources/**/schema/*.json")
private Resource[] resources;

Expanding on Luke Hutchinsons answer above, using his ClassGraph library, I was able to easily get a list of all files in a Resource folder with almost no effort at all.

Let's say that in your resource folder, you have a folder called MyImages. This is how easy it is to get a URL list of all the files in that folder:

import io.github.classgraph.ClassGraph;
import io.github.classgraph.ResourceList;
import io.github.classgraph.ScanResult;

public static LinkedList<URL> getURLList (String folder) {
    LinkedList<URL> urlList    = new LinkedList<>();
    ScanResult      scanResult = new ClassGraph().enableAllInfo().scan();
    ResourceList    resources  = scanResult.getAllResources();
    for (URL url : resources.getURLs()) {
        if (url.toString().contains(folder)) {
            urlList.addLast(url);
        }
    }
    return urlList;
}

Then you simply do this:

LinkedList<URL> myURLFileList = getURLList("MyImages");

The URLs can then be loaded into streams or use Apache's FileUtils to copy the files somewhere else like this:

String outPath = "/My/Output/Path";
for(URL url : myURLFileList) {
    FileUtils.copyURLToFile(url, new File(outPath, url.getFile()));
}

I think ClassGraph is a pretty slick library for making tasks like this very simple and easy to comprehend.

Related