Dask dataframe containing json format column

user_13162134 0
Viewed 870

I have a dask dataframe containing a column in json format, and I want to parse the column into dataframe format.

the column in json format looks like:

{"Name": {"id": 1000, "address": "ABC", ....}},,,

So I want to extract only value of "Name", and make each keys in them a column, each values a values in it, like:

id    address ...
1000  ABC
2000  DEF
3000  GHA
...   ...

I think we can read json file into dask dataframe by read_json, but how could I do that?

dask pandas python
1 Answers

The operation that you're doing appears to be embarrasingly parallel. As a result, you can write a Pandas function and then apply that function across a dask dataframe in parallel.

def f(df: pandas.DataFrame) -> pandas.DataFrame:
    ... however you would do this in Pandas

ddf = ddf.map_partitions(f)
user_6166160
Related