I want to read .csv file using pyspark in python, but I want to give just type for each column and use header(from .csv file) for column names(not names in schema). Does anybody know how to do that?
1 Answers
You always have to give column a name when defining a schema in Spark. But you can pass option inferSchema into spark.read.csv so Spark will pickup column name and infer data type automatically
Related