Converting data frame columns in category type in pyspark

Viewed 426

I have a data frame df and there I want to convert some columns into category type. Using pandas I can do it like below way:

    for col in categorical_collist:
        df[col] = df[col].astype('category')

I want to do the column conversion in pyspark. How can I do it?

I have tried using the below code in pyspark. But it is not giving my expected output during operation.

from pyspark.sql.types import StringType
for col in categorical_collist:
    df = df.withColumn(col, df[col].cast(StringType()))
0 Answers
Related