How to apply functions to a Dataframe with arrays within it, in Julia?

user_11039812 0
Viewed 631

I hope to be clear on the questions, but let me explain me better here: I have this dataframe:

m = DataFrame(
x = [1,2,3],
y = [[1,2,3],[4,5,6],[7,8,9]])

My objective is to obtain the column z which is each y-array squared plus the x-string corresponding the position in the columns. Maybe you can get it in that way:

m = DataFrame(
x = [1,2,3],
y = [[1,2,3],[4,5,6],[7,8,9]],
z = [[2,5,10],[18,27,38],[52,67,84]])

I can do it in R using this code:

m <- m %>% mutate(z = map2(x,y, ~map2_dbl(.x,.y, ~ (.x + .y^2))))

Is there something similar in Julia? Regards

julia multidimensional-array arrays
2 Answers

On DataFrames.jl master (to be released soon), you can use the transform function:

julia> using DataFrames

julia> m = DataFrame(x = [1,2,3], y = [[1,2,3],[4,5,6],[7,8,9]])
3×2 DataFrame
│ Row │ x     │ y         │
│     │ Int64 │ Array…    │
├─────┼───────┼───────────┤
│ 1   │ 1     │ [1, 2, 3] │
│ 2   │ 2     │ [4, 5, 6] │
│ 3   │ 3     │ [7, 8, 9] │

julia> transform(m, [:x, :y] => ByRow((x,y) -> y.^2 .+ x) => :z)
3×3 DataFrame
│ Row │ x     │ y         │ z            │
│     │ Int64 │ Array…    │ Array…       │
├─────┼───────┼───────────┼──────────────┤
│ 1   │ 1     │ [1, 2, 3] │ [2, 5, 10]   │
│ 2   │ 2     │ [4, 5, 6] │ [18, 27, 38] │
│ 3   │ 3     │ [7, 8, 9] │ [52, 67, 84] │
user_12695670

Something like this should work:

julia> using DataFrames

julia> m = DataFrame(
           x = [1,2,3],
           y = [[1,2,3],[4,5,6],[7,8,9]])
3×2 DataFrame
│ Row │ x     │ y         │
│     │ Int64 │ Array…    │
├─────┼───────┼───────────┤
│ 1   │ 1     │ [1, 2, 3] │
│ 2   │ 2     │ [4, 5, 6] │
│ 3   │ 3     │ [7, 8, 9] │

julia> m.z = map(eachrow(m)) do row
           row.x .+ row.y .^ 2
       end
3-element Array{Array{Int64,1},1}:
 [2, 5, 10]
 [18, 27, 38]
 [52, 67, 84]

julia> m
3×3 DataFrame
│ Row │ x     │ y         │ z            │
│     │ Int64 │ Array…    │ Array…       │
├─────┼───────┼───────────┼──────────────┤
│ 1   │ 1     │ [1, 2, 3] │ [2, 5, 10]   │
│ 2   │ 2     │ [4, 5, 6] │ [18, 27, 38] │
│ 3   │ 3     │ [7, 8, 9] │ [52, 67, 84] │

But I think it might be clearer (more readable) to avoid using an anonymous function here, and simply create a regular function instead:

       # Define the function that works on one dataframe row
julia> f(x, y) = y.^2 .+ x
f (generic function with 1 method)

julia> f(1, [1,2,3])
3-element Array{Int64,1}:
  2
  5
 10

       # And broadcast (map) it to all rows
julia> m.z = f.(m.x, m.y)
3-element Array{Array{Int64,1},1}:
 [2, 5, 10]
 [18, 27, 38]
 [52, 67, 84]

julia> m
3×3 DataFrame
│ Row │ x     │ y         │ z            │
│     │ Int64 │ Array…    │ Array…       │
├─────┼───────┼───────────┼──────────────┤
│ 1   │ 1     │ [1, 2, 3] │ [2, 5, 10]   │
│ 2   │ 2     │ [4, 5, 6] │ [18, 27, 38] │
│ 3   │ 3     │ [7, 8, 9] │ [52, 67, 84] │
user_12256070
Related