I have a data frame df and there I want to convert some columns into category type. Using pandas I can do it like below way:
for col in categorical_collist:
df[col] = df[col].astype('category')
I want to do the column conversion in pyspark. How can I do it?
I have tried using the below code in pyspark. But it is not giving my expected output during operation.
from pyspark.sql.types import StringType
for col in categorical_collist:
df = df.withColumn(col, df[col].cast(StringType()))