Failing to execute column transformation in DataFrames.jl

user_1269567 0
Viewed 58

Suppose I have the following data frame:

julia> using DataFrames

julia> df = DataFrame(id=["a", "b", "a", "b", "b"], v=[1, 1, 1, 1, 2])
5×2 DataFrame
 Row │ id      v
     │ String  Int64
─────┼───────────────
   1 │ a           1
   2 │ b           1
   3 │ a           1
   4 │ b           1
   5 │ b           2

I wanted to compute the number of unique values in column :v per group defined by column :id. I tried the following:

julia> gdf = groupby(df, :id)
GroupedDataFrame with 2 groups based on key: id
First Group (2 rows): id = "a"
 Row │ id      v
     │ String  Int64
─────┼───────────────
   1 │ a           1
   2 │ a           1
⋮
Last Group (3 rows): id = "b"
 Row │ id      v
     │ String  Int64
─────┼───────────────
   1 │ b           1
   2 │ b           1
   3 │ b           2

julia> combine(gdf, :v => x -> length(unique(x)) => :len)
2×2 DataFrame
 Row │ id      v_function
     │ String  Pair…
─────┼────────────────────
   1 │ a       1=>:len
   2 │ b       2=>:len

But it does not produce the expected result. How to fix the call to combine?

julia dataframe
1 Answers

This is a common issue. The problem is how Julia interprets your transformations specification:

julia> :v => x -> length(unique(x)) => :len
:v => var"#3#4"()

And as you can see the whole x -> length(unique(x)) => :len part, due to Julia operator precedence rules, is treated as a definition of an anonymous function. Instead you should wrap the definition of an anonymous function in parentheses like this:

julia> combine(gdf, :v => (x -> length(unique(x))) => :len)
2×2 DataFrame
 Row │ id      len
     │ String  Int64
─────┼───────────────
   1 │ a           1
   2 │ b           2

Note also that in this case you could have used function composition operator ∘ like this:

julia> combine(gdf, :v => length∘unique => :len)
2×2 DataFrame
 Row │ id      len
     │ String  Int64
─────┼───────────────
   1 │ a           1
   2 │ b           2

in which case you do not have to define an anonymous function explicitly so parentheses are not needed.

user_12695670
Related