Can CatagoricalArrays be used with Julia Dataframes to convert multiple columns from string to categories?

user_16998318 0
Viewed 231

I have a fairly large survey dataset (100+ columns and 5000 rows) that has a bunch of string variables.

I can use the following function to convert individual columns one by one,

function fix_df_column(df)
    levels = ["x"]
    Colname = categorical(df[!, :Colname]; levels, ordered = false)
    df[!, :Colname] = Colname
    #df
end

but I would like to be able to iterate across the whole dataframe and convert everything automatically.

The only code I can find relates to arrays https://dataframes.juliadata.org/stable/man/categorical/ and the only examples I can find are changes to single columns, not multiple.

Does anyone know a simpler way to achieve this?

Thanks

dataframes.jl julia
1 Answers

Yes, you can do it. Assuming that you want to convert all columns that contain strings (without missing) and you want automatic assignment of levels you can do:

transform!(df, names(df, AbstractString) .=> categorical, renamecols=false)

For example:

julia> df = DataFrame(x1=["a", "b"], x2=[1,2], x3=[missing, "x"], x4=["c", "d"])
2×4 DataFrame
 Row │ x1      x2     x3       x4
     │ String  Int64  String?  String
─────┼────────────────────────────────
   1 │ a           1  missing  c
   2 │ b           2  x        d

julia> transform!(df, names(df, AbstractString) .=> categorical, renamecols=false)
2×4 DataFrame
 Row │ x1    x2     x3       x4
     │ Cat…  Int64  String?  Cat…
─────┼────────────────────────────
   1 │ a         1  missing  c
   2 │ b         2  x        d

and you can see that only :x1 and :x4 are changed.

user_12695670
Related