My application needs to periodically scan filesystems in order to process files. Initially I was using java.nio.file.Files.walk to perform the scan, but soon enough I encountered an issue with some AccessDeniedException. Found out after some googling that Files.walk expects the explored directory tree to be accessible by the user, else it would crash and stop, which renders this function unusable for my application (it is self-hosted by a multitude of people on various kind of systems).
I changed the implementation to use java.nio.file.Files.walkFileTree instead, which seemed to work great and handled the AccessDeniedException in the user code.
However, someone recently reported that scanning time skyrocketed from a mere 12 seconds (using Files.walk) to 80 minutes (using Files.walkFileTree)! The user has around 10,000 folders and 120,000 files. It is running Windows, and the disk is using NTFS. Other users with similar number of folders/files but running Linux experience scan times under 10s, whatever the method used.
I am trying to understand what could cause the massive performance hit by using Files.walkFileTree on Windows with NTFS, but given I don't have access to a test system running Windows, I can't debug the code to understand where the time is spent.
Would you know if there are know issues around walking a file tree under Windows NTFS? And if there are some other methods I could use to perform that task? Keeping in mind that I need to handle AccessDeniedException in user code.