Fetch unique values from first columns and concat the values of other columns using spark dataframe

Viewed 12
2 Answers

You can provide these simple script in your question. It will be answered more easy.

DECLARE @Table TABLE (
    column1 sysname,
    column2 CHAR(1)
);

INSERT @Table (column1, column2)
VALUES ('D112', 'a'),
    ('D232', 'b'),
    ('D112', 'c'),
    ('D334', 'd'),
    ('D232', 'e'),
    ('D112', 'f');

AND THEN here is the answer:

SELECT t.column1, STRING_AGG(t.column2, ',')
FROM @Table AS t
GROUP BY t.column1;

Try this:

df.groupBy($"column1").agg(concat_ws(",", collect_list($"column2")))
Related