Ruby: parsing CSV from ARGF, broken?

user_3387716 0
Viewed 74

I have a file.csv:

value1,value2

that I convert to JSON using the following command:

# adding headers on the fly
echo 'key1,key2' |
ruby -rcsv -rjson -e 'puts CSV.new(ARGF,headers:true).map(&:to_hash).to_json' - file.csv

The output should be:

[{"key1":"value1","key2":"value2"}]

Which is the case with ruby-2.0 and 2.3, but with ruby-2.7, the output is:

[]

Further debugging showed that ruby-2.7 CSV lib doesn't handle ARGF well, it stops the reading after the first file in ARGF (stdin in my case).

Are newer Ruby versions also affected by this issue? Is there a more portable way to do the same magic?

csv ruby
1 Answers

I was able to reproduce this problem. It seems like the ARGF object is behaving the same way in both Ruby 2.5 and Ruby 2.7 though. Both versions give me this output:

$ echo 'key1,key2' | ruby -rcsv -rjson -e 'p ARGF.read; p ARGF.read; p ARGF.read' - file.csv
"key1,key2\n123,345\n"
nil
nil

However, there must be something weird about what the CSV library is doing that make your use case stop working.

Anyway, instead of counting on Ruby's ARGF and CSV libraries to concatenate the standard input with file.csv, you can just concatenate those two things using your shell instead, and it will work:

( echo 'key1,key2'; cat file.csv ) | ruby -rcsv -e 'puts CSV(ARGF,headers:true).map(&:to_hash)'

or

ruby -rcsv -e 'puts CSV(ARGF,headers:true).map(&:to_hash)' <( echo 'key1,key2'; cat file.csv )
David_Grayson_281280
Related