Importing csv file into J and using them as variable

user_8644706 0
Viewed 97

I saved this data (20 vectors v)into csv file like this 

v=:<"1 (? 20 2 $ 20)
makecsv v
v writecsv jpath'~temp/position.csv'
]vcsv =: freads jpath '~temp/position.csv'
fixcsv vcsv

, and I could import the csv file by

readcsv jpath '~temp/position.csv'

However, it doesn't give same result if I name it as

w=: readcsv jpath '~temp/position.csv'
diff=: ([{]) ,. ]

0 diff v
0 diff w

Actually, 0 diff w gives a length error

Is there any other approach should I use to have same results from both v(original) and w(imported csv data)?

Thank you!

j
2 Answers

I'm a J beginner so you may get a better answer later, but poking at it I think I have found something.

First, the tables/csv addon docs state that readcsv "Reads csv file into a boxed array," emphasis mine, while writecsv "Writes an array to a csv file." In other words, readcsv and writecsv are not symmetric operations. And the shapes of the values seem to confirm that:

   $ w
1 20
   $ v
20

This is also why the diff works for v but not w. If you simply unbox the result, it seems to work better:

0 diff 0 { w
┌───┬─────┐
│1 3│1 3  │
├───┼─────┤
 ...
├───┼─────┤
│1 3│5 8  │
└───┴─────┘

However, the shapes are still not exactly the same:

   $ > v
20 2
   $ > 0 { w
20 5

I think this is because readcsv doesn't know that your values are numeric; you probably need to throw a ". in there somewhere to decode them.

user_8128180

When you write the CSV file, you just have a bunch of ASCII characters. In this case, you've got numbers, spaces, and commas.

When you read the CSV, J has no guarantees about the format or contents. fixcsv gets your commas and line-breaks translated into a grid of cells, but J boxes it all to be safe, because it's a bunch of variable-length ASCII strings.

If you want to get back to v, you have two things you need to do. The first is to get the dimensions right. CSV files, pretty much by definition, are two-dimensional. If you change your example to write a two-dimensional array to the CSV, you'll find that you have the same shape after fixcsv readcsv.

u =: 4 5 $ v
   u writecsv jpath'~temp/position.csv'
104
   ] t =: fixcsv freads jpath '~temp/position.csv'
┌────┬─────┬────┬────┬─────┐
│9 11│1 4  │8 3 │3 12│5 4  │
├────┼─────┼────┼────┼─────┤
│7 11│10 11│9 10│0 8 │6 16 │
├────┼─────┼────┼────┼─────┤
│13 8│17 12│13 2│5 19│17 14│
├────┼─────┼────┼────┼─────┤
│2 15│19 10│3 1 │12 7│14 13│
└────┴─────┴────┴────┴─────┘
   $ v
20
   $ u
4 5
   $ t
4 5

If you're definitely dealing with a one-dimensional list (albeit of boxed number pairs), then you can Ravel (,) what you read to get it down to one dimension.

   $ w
1 20
   $ , w
20

Once you have them in the same shape, you need to convert the ASCII text into number arrays. Do that with Numbers (".).

   10 * > {. v
90 110
   10 * > {. , w
|domain error
|   10    *>{.,w
   'a' , > {. , w
a9 11
   10 * _ ". > {. , w
90 110
user_8953230
Related