Julia. Summarise one column into a new DataFrame with multiple columns

user_380403 0
Viewed 197

I need to group a dataframe by one variable and then summarising it by adding the number or rows (I can already do this) and number of columns relative to .25, .5, .75 quantiles of another variable.

In R I would do e.g.:

    iris %>%
       group_by(Species) %>%
       summarise(
          quantile(Sepal.Length, c(.25, .75)) %>%
             matrix(nrow = 1) %>%
             as.data.frame() %>%
             setNames(paste0("Sepal.Length", c(.25, .75)))
    )

What would be a concise way to write this in Julia using DataFrames and DataFrameMeta? If there's a solution to apply this to multiple columns at once even better.

The closest solution I could find in Julia was:

groupby(iris, :Species) |>
   x -> combine(x, :Sepal.Length => x -> [[map(p -> quantile(x, p), (Q25 = 0.25, Q75 = 0.75))] |> DataFrame])

but it just encapsulates the dataframe into a cell, while it should spread it into multiple columns.

data-wrangling julia dataframe
1 Answers

This is shortest I can currently propose you:

combine(groupby(iris, :Species), :SepalLength => (x -> (quantile(x, [0.25, 0.75]))') => [:q25, :q75])

or similarly

combine(groupby(iris, :Species), :SepalLength => (x -> [quantile(x, [0.25, 0.75])]) => [:25, :q75])

or

combine(groupby(iris, :Species), :SepalLength .=> [x -> quantile(x, q) for q in [0.25, 0.75]] .=> [:q25, :q75])

But even your original solution seemed a bit shorter than R. Also I would rewrite it as:

combine(groupby(iris, :Species), :SepalLength => (x -> map(p -> quantile(x, p), (Q25=0.25, Q75=0.75))) => AsTable)

which seems a bit cleaner.

Now if you wanted to process multiple columns you could do (BTW - how would you do this in R?):

combine(groupby(iris, :Species), [n => (x -> (quantile(x, [0.25, 0.75]))') => [n*"_q25", n*"_q75"] 
                                  for n in ["SepalLength",  "SepalWidth", "PetalLength", "PetalWidth"]])
user_12695670
Related