R data.table fwrite to fread space delimiter and empties

Viewed 431

I am having problems using fread with " " as delimiter and interspersed blank values. For example, this:

dt <- data.table(1:5,1:5,1:5) #make a simple table
dt[3,"V2" := NA] #add a blank in the middle to illustrate the problem
fwrite(dt, file = "dt.csv", sep = " ") #save to file
dt <- fread("dt.csv", sep = " ") #try to retrieve

The fread fails with: "Stopped early on line 4. Expected 3 fields but found 2." The problem seems to be that with the NA value in the middle column, fwrite gives value|space|space|value, then fread doesn't recognize the implied blank value in the middle.

I understand it would be simple to use another delimiter in the first place. However, is it possible to get fread to reproduce the original dt here?

EDIT WITH A READ-SIDE SOLUTION:

I found the same question here. It's a bit confusing because it gives a solution, but then the solution later stopped working. On pursuing some other leads the closest I've now found to a read-side solution with fread() is with a Unix command like this:

dt <- fread(cmd="wsl sed -r 's/ /,/g' dt.csv") #converts spaces to commas on the way in

On Windows 10 I had to do some trial and error to get my system to run Unix commands. The "wsl" part seems to depend on the system. This video was helpful, and I used the first method he describes there. This and this question provides a bit more on sed with fread. The latter says sed comes with rTools, although I did not try that.

1 Answers

Maybe export NA as something other than "" by default

Here I use @

library(data.table)
dt <- data.table(1:5,1:5,1:5) #make a simple table
dt[3,"V2" := NA] #add a blank in the middle to illustrate the problem
fwrite(dt, file = "dt.csv", sep = " ", na="@") #save to file
dt <- fread("dt.csv", sep = " ",na.strings = "@") #try to retrieve
Related