An array of dictionaries into a DataFrame at ONE go in Julia

user_10907660 0
Viewed 1385

I have an array of dictionaries as follows:

julia> data  
2-element Array{Any,1}:  
 Dict{String,Any}("ratio1"=>1.36233,"time"=>"2014-06-19T15:47:40.000000Z","ratio2"=>1.36243)
 Dict{String,Any}("ratio1"=>1.3623,"time"=>"2014-06-19T15:48:00.000000Z","ratio2"=>1.36245)

How can I pop this into a DataFrame at ONE go without looping through each dictionary and key one by one, so that I have a Dataframe like below: 

2×3 DataFrame
│ Row │ ratio1  │ ratio2  │ time                        │
│     │ Float64 │ Float64 │ String                      │
├─────┼─────────┼─────────┼─────────────────────────────┤
│ 1   │ 1.36233 │ 1.36243 │ 2014-06-19T15:47:40.000000Z │
│ 2   │ 1.3623  │ 1.36245 │ 2014-06-19T15:48:00.000000Z │
julia
1 Answers

One way would be

julia> vcat(DataFrame.(data)...)
2×3 DataFrame
│ Row │ ratio1  │ ratio2  │ time                        │
│     │ Float64 │ Float64 │ String                      │
├─────┼─────────┼─────────┼─────────────────────────────┤
│ 1   │ 1.36233 │ 1.36243 │ 2014-06-19T15:47:40.000000Z │
│ 2   │ 1.3623  │ 1.36245 │ 2014-06-19T15:48:00.000000Z │

julia> @btime vcat(DataFrame.($data)...)
  31.146 μs (157 allocations: 12.19 KiB)

i.e. convert every Dict to a DataFrame and concatenate them.

More detailed explanation:

  • DataFrame(somedict) is a constructor call which creates a DataFrame from a Dict

  • DataFrame.(arrayofdicts): The dot here broadcasts the constructor call such that all contained Dicts get converted to DataFrames and we obtain an array of DataFrames. See Dot Syntax for Vectorizing Functions in the Julia docs for more information.

  • vcat(DataFrame.(arrayofdicts)...): We now splat our array of DataFrames into the vcat function, which concatenates them vertically (rows).

user_23656750
Related