I recently started working with Databricks and I am new to Pyspark. I am trying to display a tidy and understandable dataset from a text file in pyspark.
Here is the code snippet:
# File location and type
file_location = "/FileStore/tables/2014_GE_By_Precinct.txt"
file_type = "txt"
# The applied options are for txt files. For other file types, these will be ignored.
df = spark.read.option("inferSchema", "true") \
.option("header", "true") \
.csv(file_location)
df.show(truncate=False)
Here is the result I am getting:
I want the dataframe to be displayed in a way so that I can scroll it horizontally and all my column headers fit in one top line instead of a few of them coming in the next line and making it hard to understand which column header represents which column.