Java 17 path exists() taking an extraordinarily long time when file exists

Question

I'm using Java 17 on Windows 10. I have a set of directories in this form:

data/dir-0
data/dir-1
data/dir-2
…
data/dir-999

I have a loop over the integers 0 to 999 with variable i. Within each loop, I just check to see if there is a foo.bar file inside the subdirectory:

Path foobarFile = dataDir.resolve("dir-"+i);
if(File.exists(fooBarFile)) {
  //do something; currently does nothing
}

If all the data/dir-* directories are empty, them boom, the entire loop ends in a flash (some small fraction of a second).

But if some of the data/dir-* directories contain a foo.bar file, the loop speeds through the empty directories until it actually finds a foo.bar file, and then suddenly each exists(fooBarFile) call takes almost half a second! (And each call takes the same amount of time, as if there was some clock counting.)

Then it gets even weirder: if I run the program again with no changes, it somehow "remembers where it left off", and will speed through all the directories until it gets to one it hasn't checked before. Then each exists(fooBarFile) call takes almost a second.

Note that this happens both within Eclipse and from the command line.

Why does exists(fooBarFile) take so long when the file actually exists? And how could it "remember" which ones it checked between running the application? I'm flummoxed.

Update: Apparently here is how Java 17 checks to see if a Windows path exists, inside sun.nio.fs.checkReadAccess(WindowsPath file):


try {
  Set opts = Collections.emptySet();
  FileChannel fc = WindowsChannelFactory
      .newFileChannel(file.getPathForWin32Calls(),
          file.getPathForPermissionCheck(), opts, 0L);
  fc.close();
} catch (WindowsException exc) {
…

Yes, it appears that Java tells if a file exists by opening up a channel to the file!!

Tell me it isn't so. I just need to know if a file is there so I'll know whether it needs to be generated or not. I don't need to check for read access or anything. Isn't there a more efficient way? And besides, surely opening and closing a channel shouldn't take a whole second on a recent machine with an SSD, should it??

user_421049 · Answer

This seems to be related to Windows Defender. I went to Windows Security Center > Virus & threat protection > Virus & threat protection settings and added the data directory to the exclusions. Suddenly my program screamed through all the directories.

Apparently Java's approach for checking that a file exists (by opening the file) coupled with Windows Defender slows file existence checking to a halt!!

(I just verified this by removing the Windows Defender exclusion; the loop screeched to a halt when I ran it immediately after doing that.)

This is not a good thing. This is not a good thing at all. Neither Java nor Windows Defender seem to be doing the ideal thing here, either.

(I've also noticed that the first time I do a Maven build in a directory, it's horribly slow after a reboot until I've built the project once. I wonder if adding the project directory to the Windows Defender exclusion list would help …)

As Luca Scarcia suggested, File.exist() uses a different technique and amazingly seems to fly through all the directories, even without a Windows Defender exclusion.

Why, why, why is Java using this brute-force open-the-file to check for existence with Java NIO Files.exists(Path)? This is a problem.

Java 17 path exists() taking an extraordinarily long time when file exists

1 Answers