gnuplot: How to get a file list with non-ANSI characters?

Question

I would like to load several files into gnuplot (under Windows). For this I use the command system(). Check the follow minimal example. This works fine as long as there are only ANSI characters in the filenames.

Well, the greek and kyrillic alphabet might be extreme cases but characters, like ä,ö,ü,ß,é,à,ñ, ... would be nice to be handled correctly.

Of course, I could:

rename the files to contain only ANSI-characters (not really an option)
create a file of filenames with external tools and save it as utf-8 file and load this file list into gnuplot (to be avoided if possible)
create such a file of filenames with gnuplot using set print, however, gnuplot seems to save the file as ANSI encoded.

Is there a direct solution with gnuplot to handle these special characters?

Code:

### get a file list from a directory (Windows)
reset session
set encoding utf8

PATH = 'CountryTest\'
GetFileList(p,f) = sprintf('dir /b "%s%s"', p, f)

FILES = system(GetFileList(PATH,'*.dat'))
print FILES

set print "Countries.txt"
    print FILES
set print
### end of code

Result1: (gnuplot 5.2.8 console, encoding utf8)

Belgi�.dat
C�te d'Ivoire.dat
Espa�a.dat
L�tzebuerg.dat
M�xico.dat
Panam�.dat
Per�.dat
Rom�nia.dat
T�rkiye.dat
�ire.dat
�sland.dat
�sterreich.dat
Cesko.dat
?????.dat
??????.dat

Result2: (file Countries.txt created by gnuplot, Notepad++ tells me that the file is ANSI encoded)

Belgi‰.dat
C“te d'Ivoire.dat
Espa¤a.dat
L‰tzebuerg.dat
M‚xico.dat
Panam .dat
Per£.dat
Romƒnia.dat
Trkiye.dat
ire.dat
Ösland.dat
™sterreich.dat
Cesko.dat
?????.dat
??????.dat

Update:

If I type the following in the Windows console:

chcp 65001 && dir /b "CountryTest\*.dat" > "CountryTest\Countries.txt"

I get the file Countries.txt:

België.dat
Côte d’Ivoire.dat
España.dat
Lëtzebuerg.dat
México.dat
Panamá.dat
Perú.dat
România.dat
Türkiye.dat
Éire.dat
Ísland.dat
Österreich.dat
Česko.dat
Ελλάς.dat
Россия.dat

However, if I use the same command in a gnuplot script:

system('chcp 65001 && dir /b "CountryTest\*.dat" > "CountryTest\Countries.txt"')

I get this file Countries.txt:

Belgi‰.dat
C“te d'Ivoire.dat
Espa¤a.dat
L‰tzebuerg.dat
M‚xico.dat
Panam .dat
Per£.dat
Romƒnia.dat
Trkiye.dat
ire.dat
Ösland.dat
™sterreich.dat
Cesko.dat
?????.dat
??????.dat

So, what is going wrong with gnuplot's system() command?

user_7295599 · Answer

Thanks to user @ItWasn'tMe on superuser, I ended up with a solution which probably cannot be improved further unless there will be a direct implementation in gnuplot.

Starting point: (a subdirectory CountryTest\ with the following file names)

België.dat
Côte d’Ivoire.dat
España.dat
Lëtzebuerg.dat
México.dat
Panamá.dat
Perú.dat
România.dat
Türkiye.dat
Éire.dat
Ísland.dat
Österreich.dat
Česko.dat
Ελλάς.dat
Россия.dat

Code: (tested with Windows 10 and gnuplot >=5.2.0)

### get UTF8 file list into a datablock (Windows)
reset session

# get UTF8 filenames of a filtered directory (Windows only)
GetFileList(p,f,d) = sprintf('nul & cmd.exe /c dir /b "%s%s"', d, p, f)

PATH = 'CountryTest\'
EXPR = '*.dat'

load GetFileList(PATH, EXPR, '$myFiles')

set offsets 1,1,1,1

plot $myFiles u 0:0:($Data[$0+1]) w labels notitle
### end of code

Result:

gnuplot: How to get a file list with non-ANSI characters?

1 Answers