Unexpected Error comparing directories in Win 7

Viewed 584

I need to compare for equality two directories that hold a large number of files (eg. comp d1\*.* d2\*.*). A for loop comparing the files individually is far too slow. I only care if all files exist and are equal; in other words, I only care about the comp.exe return code.

However when a file exists that begins with a period I get an unexpected error. For example, if I do the following:

md d1
echo x >d1\.file
md d2
echo x >d2\.file
comp d1\*.* d2\*.*

it produces:

d1\.file      d2\*.*
Could not expand second file name so as to match first

Replacing comp with fc produces the identical result. I have no idea why this is happening -- the directories should be entirely equal -- and wonder if there is a workaround.

2 Answers

Open a command prompt window and run comp /? or fc /? and read the output short help. Or read the two referenced Microsoft help pages with examples from Microsoft's command line reference.

We have to clarify some terms to understand the problem.

The name of a file on Windows consists of two parts:

  1. The file name being everything left to last . in name of file.
  2. The file extension being last . and everything right to it in name of file.

Well, the last . is in real the separator between file name and file extension. Therefore the string after last . is in real the file extension. But the dot/point must be also belonging to either file name or file extension to build on simply concatenation of both parts the name of file. For that reason the last dot is added to file extension.

So on Windows a file with name of file being .file has an empty file name and .file as file extension which can be seen on running in a command prompt window:

@for %I in (.file) do @echo File name="%~nI" & echo File extension="%~xI"

The output is:

File name=""
File extension=".file"

On Unix/Linux there is no such definition of name of file consisting of two parts although many files on Unix/Linux have also name.ext syntax because this makes it easier for people to get an information about type of file (png, jpg, sh, txt, ...).

All Windows file systems support the attribute hidden to mark a file or directory as hidden. Unix/Linux file systems do not have such an attribute. On Unix/Linux a directory or file is interpreted as hidden when its name starts with .. For that reason a file like .file is interpreted on Unix/Linux as hidden file with name file. .htaccess is a very common hidden file on Linux web servers of which name is in real htaccess while the dot at begin marks this file as hidden on Unix/Linux.

Okay. So what is the problem here on comparing the directories d1 and d2 containing both only one file with .file as name of file?

The two files have no file name. But both compare commands need on usage of wildcards the file name parts to identify those two files from both directories to compare. That's how those two comparison commands on Windows are designed for comparing multiple files in two directories.

Working are:

comp d1\.file d2\.file
fc d1\.file d2\.file

But a comparison with wildcards does not work if one file in one of the two directories matching the wildcard pattern have an empty file name. The two console applications comp.exe and fc.exe in %SystemRoot%\System32 fail in this special case to find the matching file in the other directory also having no file name, just a matching file extension.

The used command line comp d1\*.* d2\*.* could be also written as comp d1\* d2\* because Windows interprets *.* always like just *. This can be seen on running dir *.* which finds also folders and files not containing a . at all.

On using comp d1\*.* d2\*.* the console application COMP searches first for any file in directory d1 and in this case finds the file .file. It splits the name of this file up into file name and file extension. The file name is an empty string with length 0.

COMP is written to search in other directory d2 for a file with same file name as the found file in first directory has with the file extension specified on command line as second argument. In this case the specified file extension on command line is also * which means any file extension.

So COMP should search for nothing.* or more precisely search for .* in folder d2 because the file with name of file .file has no file name.

What does dir .* output in comparison to dir * or dir *.*?

dir .* outputs just file and folder names starting with a dot while dir * and dir *.* output both all files and folders in current directory.

So COMP should work to find file .file in d2.

But if COMP finds a file with an empty file name, it does not even run this search in other directory and instead outputs the error message.

Interesting is that not working are in this special use case:

comp d1\*      d2\*
comp d1\*.*    d2\*.*
comp d1\.file  d2\*
comp d1\.file  d2\*.*
comp d1\.file  d2\*.file

But working are in this special use case:

comp d1\*      d2\.file
comp d1\*.*    d2\.file
comp d1\*.file d2\.file

This is (for me) a quite clear indication for a special use case not taken into account by the developers who wrote the code of COMP and FC. Users rate usually such a not correct handled use case by an application as BUG.

The simple solution to workaround this bug of COMP and FC is to use a different directory comparison tool like Beyond Compare, UltraCompare, etc. which can handle this special case correct.

A possible workaround could be temporarily renaming files starting with a dot by inserting ### at begin, running the directory comparison, and renaming the files starting with ### back to their original names.

@echo off
ren d1\.* ###.* 2>nul
ren d2\.* ###.* 2>nul
echo N | %SystemRoot%\System32\comp.exe d1\* d2\* >nul 2>&1
if errorlevel 1 ( set "Result=different" ) else set "Result=identical"
echo Folders d1 and d2 are %Result%.
ren d1\###.* .* 2>nul
ren d2\###.* .* 2>nul

Of course this simple workaround solution does not work correct if there are files with ### at begin of name of file already present in one of the two directories.

As you noticed, fc and comp have problems with filenames without name ("starting with a dot") and wildcard usage. There is no problem, if you don't use wildcards. This can be done with a simple for loop:

@echo off 
setlocal
for %%a in (d1\*) do (
  fc "d1\%%~nxa" "d2\%%~nxa" >nul 2>&1
  if errorlevel 1 set /a fail+=1
)
echo Failed: %fail%
if %fail% == 0 echo all ok. 

(I recommend fc, because it doesn't ask for more files to compare; All output is discarded by >nul 2>&1)

Related