PySpark read in multiple files CSV or TSV

Viewed 18

I'm trying to load all the files in a folder. They have they same schema, but sometimes have a different delimiter (ie Usually CSV, but occasionally tab separated)

Is there a way to pass in two delimiters? Being specific I don't want a two character delimiter "||", but to be able to treat multiple delimiters the same way.

I'm letting it infer the schema. Commas work, but tabbed rows just end up in the first column.

0 Answers
Related