Hello,
This is my first posting here. I use excellent R data.table package. I need to import a file without comment lines, but I don't see any option in fread() to get rid of comment lines which are spread across the file, not only at the beginning of the file. To simplify - the file test.txt consists of 4 lines, comment lines begin with "#":
#A
A AA
A A#A
#A
I import data with fread() and then get rid of comment lines with grep (^#); everything works. There is also an option to use grep inside fread() as a command line call instead a single file name. (For the record, I working in Windows, thus I have grep.exe in my project folder.) Grep works with simple regular expressions as expected when I call it from R:
> system("grep # test.txt")
#A
A A#A
#A
> system("grep ^# test.txt")
#A
#A
But it ignores the beginning of line command "^" when called as a system command inside fread() function:
> fread("grep # test.txt", sep = "\t", header = FALSE, fill = TRUE)
V1 V2
1: #A
2: A A#A
3: #A
> fread("grep ^# test.txt", sep = "\t", header = FALSE, fill = TRUE)
V1 V2
1: #A
2: A A#A
3: #A
Thus, grep.exe as well as grep() in R are working as expected, but grep.exe called from fread() ignores beginning of line command (didn't try other regex). What is wrong here?