Reading TXT files in zipped folder with fread without unzipping in r

user_12434059 0
Viewed 67

I am trying to read TXT files in a compressed folder with two folders inside.

Data (main folder)

  • river (subfolder)
  • treatment (subfolder)

Inside river I have many files in txt format and I am using the following code to read them.

df <- fread(cmd = 'unzip -p Data.zip river//Main.txt',
                      select = c(1,2,5,25),
                      sep = '|',
                      header = FALSE,
                      stringsAsFactors = FALSE,       
                      quote = "")

However, this results in 0 observations and I am getting the following error:

caution: filename not matched: river//Main.txt

data-wrangling data.table fread r
1 Answers

I suspect all you need to do is remove the double-slash from your path.

df <- fread(cmd = 'unzip -p Data.zip river/Main.txt',
                      select = c(1,2,5,25),
                      sep = '|',
                      header = FALSE,
                      stringsAsFactors = FALSE,       
                      quote = "")

Reproducible example:

dir.create("river")
write.table(mtcars, "river/mt.txt", row.names = FALSE, sep = "|")
system("zip -r quux.zip river")
#   adding: river/ (stored 0%)
#   adding: river/mt.txt (deflated 58%)
# [1] 0
system("unzip -v quux.zip")
# Archive:  quux.zip
#  Length   Method    Size  Cmpr    Date    Time   CRC-32   Name
# --------  ------  ------- ---- ---------- ----- --------  ----
#        0  Stored        0   0% 2022-06-13 11:56 00000000  river/
#     1336  Defl:N      555  59% 2022-06-13 11:56 2c78d73f  river/mt.txt
# --------          -------  ---                            -------
#     1336              555  59%                            2 files
# [1] 0

With double-slashes:

fread(cmd = "unzip -p quux.zip river//mt.txt", sep = "|")
# caution: filename not matched:  river//mt.txt
# Warning in (if (.Platform$OS.type == "unix") system else shell)(paste0("(",  :
#   '(unzip -p quux.zip river//mt.txt) > C:\Users\r2\AppData\Local\Temp\Rtmpw5tEvb\file34704da67edb' execution failed with error code 11
# Warning in fread(cmd = "unzip -p quux.zip river//mt.txt", sep = "|") :
#   File 'C:\Users\r2\AppData\Local\Temp\Rtmpw5tEvb\file34704da67edb' has size 0. Returning a NULL data.table.
# Null data.table (0 rows and 0 cols)

Fix that and it works:

fread(cmd = "unzip -p quux.zip river/mt.txt", sep = "|")
#       mpg   cyl  disp    hp  drat    wt  qsec    vs    am  gear  carb
#     <num> <int> <num> <int> <num> <num> <num> <int> <int> <int> <int>
#  1:  21.0     6 160.0   110  3.90 2.620 16.46     0     1     4     4
#  2:  21.0     6 160.0   110  3.90 2.875 17.02     0     1     4     4
#  3:  22.8     4 108.0    93  3.85 2.320 18.61     1     1     4     1
#  4:  21.4     6 258.0   110  3.08 3.215 19.44     1     0     3     1
#  5:  18.7     8 360.0   175  3.15 3.440 17.02     0     0     3     2
#  6:  18.1     6 225.0   105  2.76 3.460 20.22     1     0     3     1
#  7:  14.3     8 360.0   245  3.21 3.570 15.84     0     0     3     4
#  8:  24.4     4 146.7    62  3.69 3.190 20.00     1     0     4     2
#  9:  22.8     4 140.8    95  3.92 3.150 22.90     1     0     4     2
# 10:  19.2     6 167.6   123  3.92 3.440 18.30     1     0     4     4
# ---                                                                  
# 23:  15.2     8 304.0   150  3.15 3.435 17.30     0     0     3     2
# 24:  13.3     8 350.0   245  3.73 3.840 15.41     0     0     3     4
# 25:  19.2     8 400.0   175  3.08 3.845 17.05     0     0     3     2
# 26:  27.3     4  79.0    66  4.08 1.935 18.90     1     1     4     1
# 27:  26.0     4 120.3    91  4.43 2.140 16.70     0     1     5     2
# 28:  30.4     4  95.1   113  3.77 1.513 16.90     1     1     5     2
# 29:  15.8     8 351.0   264  4.22 3.170 14.50     0     1     5     4
# 30:  19.7     6 145.0   175  3.62 2.770 15.50     0     1     5     6
# 31:  15.0     8 301.0   335  3.54 3.570 14.60     0     1     5     8
# 32:  21.4     4 121.0   109  4.11 2.780 18.60     1     1     4     2
user_33582720
Related