Turn list of dask Series into a dask DataFrame

Viewed 211

I have a list that contains several dask Series. Each Series is boolean and has the same length. Here is an extract of three elements, just to show them.

[Dask Series Structure:
npartitions=15
    bool
     ...
    ... 
     ...
     ...
Name: Sequence, dtype: bool
Dask Name: eq, 75 tasks, Dask Series Structure:
npartitions=15
    bool
     ...
    ... 
     ...
     ...
Name: Sequence, dtype: bool
Dask Name: eq, 75 tasks, Dask Series Structure:
npartitions=15
    bool
     ...
    ... 
     ...
     ...
Name: Sequence, dtype: bool
Dask Name: eq, 75 tasks]

I turn this list into a dask DataFrame, whereby each series will represent a different column. I do so with dd.concat(List_of_series, axis=1, interleave_partitions=True).

The command works, but I get the following UserWarning:

/home/schmat/.local/lib/python3.6/site-packages/dask/dataframe/multi.py:1056: UserWarning: Concatenating dataframes with unknown divisions.
We're assuming that the indexes of each dataframes are 
 aligned. This assumption is not generally safe.
  "Concatenating dataframes with unknown divisions.\n"

How can I get rid of this warning? The resulting concatenation is exactly what I want, when I compute() it.

1 Answers
Related