Converting array to DataFrame or Saving to CSV in Julia

user_7601517 0
Viewed 305

My data structure looks similar to

tdata = Array{Int64,1}[]
# After 1st collection, push the first batch of data
push!(tdata, [1, 2, 3, 4, 5])
# After 2nd collection, push this batch of data
push!(tdata, [11, 12, 13, 14, 15])

Therefore, my data is

> tdata
2-element Array{Array{Int64,1},1}:
[1, 2, 3, 4, 5]
[11, 12, 13, 14, 15]

When I tried to convert this to a DataFrame,

> convert(DataFrame, tdata)
ERROR: MethodError: Cannot `convert` an object of type Array{Array{Int64,1},1} to an object of type DataFrame

while I was hoping for similar to

2×5 DataFrame
 Row │ c1     c2     c3     c4     c5    
     │ Int64  Int64  Int64  Int64  Int64 
─────┼───────────────────────────────────
   1 │     1      2      3      4      5
   2 │    11     12     13     14     15

Alternatively, I tried to save it to .CSV, but

> CSV.write("",tdata)
ERROR: ArgumentError: 'Array{Array{Int64,1},1}' iterates 'Array{Int64,1}' values, which doesn't satisfy the Tables.jl `AbstractRow` interface

Clearly, I have some misunderstanding of the data structure I have. Any suggestion is apprecitaed!

julia dataframe csv arrays
1 Answers

Either do this:

julia> using SplitApplyCombine

julia> DataFrame(invert(tdata), :auto)
2×5 DataFrame
 Row │ x1     x2     x3     x4     x5    
     │ Int64  Int64  Int64  Int64  Int64 
─────┼───────────────────────────────────
   1 │     1      2      3      4      5
   2 │    11     12     13     14     15

or this:

julia> DataFrame(transpose(hcat(tdata...)), :auto)
2×5 DataFrame
 Row │ x1     x2     x3     x4     x5    
     │ Int64  Int64  Int64  Int64  Int64 
─────┼───────────────────────────────────
   1 │     1      2      3      4      5
   2 │    11     12     13     14     15

or this:

julia> DataFrame(vcat(transpose(tdata)...), :auto)
2×5 DataFrame
 Row │ x1     x2     x3     x4     x5    
     │ Int64  Int64  Int64  Int64  Int64 
─────┼───────────────────────────────────
   1 │     1      2      3      4      5
   2 │    11     12     13     14     15

or this:

julia> df = DataFrame(["c$i" => Int[] for i in 1:5])
0×5 DataFrame

julia> foreach(x -> push!(df, x), tdata)

julia> df
2×5 DataFrame
 Row │ c1     c2     c3     c4     c5    
     │ Int64  Int64  Int64  Int64  Int64 
─────┼───────────────────────────────────
   1 │     1      2      3      4      5
   2 │    11     12     13     14     15

The challenge with your data is that you want vectors to be rows of the data frame, and normally vectors are treated as columns of a data frame.

user_12695670
Related