Julia convert dataframe column types

user_9127264 0
Viewed 468

No prior Julia knowledge, trying to fit some dataframe into MixedModels package's function

using CSV
data = CSV.read("some_file.csv")
typeof(data.log_volume)

Running this code where log_volume is variable of floating point numbers I receive next:

julia> typeof(data.log_volume)
CSV.Column{Float64,Float64}

What I want to is to read data from csv file (or convert data received with mentioned above method) so the answer would be following:

julia> typeof(data.log_volume)
Array{Float64,1}

Once again, I have no idea how types in Julia works. What I see is I need either some way of how to convert dataframe or another method of reading data from .csv. Dataframe is big, like 83k observations and 80 variables.

julia dataframe
1 Answers

You should use:

DataFrame(CSV.File("some_file.csv"))

in your case.

Now to explain what is happening have a look at help of CSV.read:

help?> CSV.read
  CSV.read(source; copycols::Bool=false, kwargs...) => DataFrame

  Parses a delimited file into a DataFrame. copycols determines whether a copy of columns should be made when creating the DataFrame; by default, no copy
  is made, and the DataFrame is built with immutable, read-only CSV.Column vectors. If mutable operations are needed on the DataFrame columns, set
  copycols=true.

  CSV.read supports the same keyword arguments as CSV.File.

So as you can see CSV.read uses its special internal column type format for performance reasons (with a drawback that it is immutable and non-standard).

The reason why I recommend to use DataFrame(CSV.File("some_file.csv")) rather than CSV.File("some_file.csv", copycols=true) is that in the future probably CSV.read function might be removed from the package, so using CSV.File is a future-safe approach.

user_12695670
Related