koalas Column assignment doesn't support type ndarray

user_8549611 0
Viewed 855

All - I am trying to add a new column to an existing koalas dataframe but it fails with the error above. The value I am assigning with is an np array. Am I missing something at all? This works well with pandas.

import databricks.koalas as ks
from sklearn.datasets import load_iris
iris = load_iris()
df = ks.DataFrame(data=iris.data, columns=iris.feature_names)
# works so far!!

df["target"] = iris.target ## this errors out!

TypeError: Column assignment doesn't support type ndarray

Am I missing anything here?

thanks.

spark-koalas pandas
2 Answers

Unfortunately, even df.assign did not solve the problem and I was getting the same error:

I had to do this:

ks.reset_option('compute.ops_on_diff_frames')
# convert target to a koalas series so that it can be assigned to the dataframe as a column
ks_series = ks.Series(iris.target)
df["target"] = ks_series
ks.reset_option('compute.ops_on_diff_frames')
user_85496110

My bad:

I misread where and what the issue was. Try the following:

...
df.assign(target=iris.target)

Could you try the following:

...
df = ks.DataFrame(data=iris.data, columns=list(iris.feature_names))
...

Looking into the load_iris documentation, they make a not of converting the returned array into a list.

user_138422610
Related