How do I expand the output display to see more columns of a Pandas DataFrame?

Viewed 1233569

Is there a way to widen the display of output in either interactive or script-execution mode?

Specifically, I am using the describe() function on a Pandas DataFrame. When the DataFrame is five columns (labels) wide, I get the descriptive statistics that I want. However, if the DataFrame has any more columns, the statistics are suppressed and something like this is returned:

>> Index: 8 entries, count to max
>> Data columns:
>> x1          8  non-null values
>> x2          8  non-null values
>> x3          8  non-null values
>> x4          8  non-null values
>> x5          8  non-null values
>> x6          8  non-null values
>> x7          8  non-null values

The "8" value is given whether there are 6 or 7 columns. What does the "8" refer to?

I have already tried dragging the IDLE window larger, as well as increasing the "Configure IDLE" width options, to no avail.

21 Answers

Only using these three lines worked for me:

pd.set_option('display.max_columns', None)
pd.set_option('display.expand_frame_repr', False)
pd.set_option('max_colwidth', -1)

It was for Anaconda, Python 3.6.5, Pandas 0.23.0, and Visual Studio Code 1.26.

I used these settings when the scale of the data was high.

# Environment settings: 
pd.set_option('display.max_column', None)
pd.set_option('display.max_rows', None)
pd.set_option('display.max_seq_items', None)
pd.set_option('display.max_colwidth', 500)
pd.set_option('expand_frame_repr', True)

You can refer to the documentation here.

The below line is enough to display all columns from a dataframe.

pd.set_option('display.max_columns', None)
import pandas as pd
pd.set_option('display.max_columns', 100)
pd.set_option('display.width', 1000)

SentenceA = "William likes Piano and Piano likes William"
SentenceB = "Sara likes Guitar"
SentenceC = "Mamoosh likes Piano"
SentenceD = "William is a CS Student"
SentenceE = "Sara is kind"
SentenceF = "Mamoosh is kind"


bowA = SentenceA.split(" ")
bowB = SentenceB.split(" ")
bowC = SentenceC.split(" ")
bowD = SentenceD.split(" ")
bowE = SentenceE.split(" ")
bowF = SentenceF.split(" ")

# Creating a set consisting of all words

wordSet = set(bowA).union(set(bowB)).union(set(bowC)).union(set(bowD)).union(set(bowE)).union(set(bowF))
print("Set of all words is: ", wordSet)

# Initiating dictionary with 0 value for all BOWs

wordDictA = dict.fromkeys(wordSet, 0)
wordDictB = dict.fromkeys(wordSet, 0)
wordDictC = dict.fromkeys(wordSet, 0)
wordDictD = dict.fromkeys(wordSet, 0)
wordDictE = dict.fromkeys(wordSet, 0)
wordDictF = dict.fromkeys(wordSet, 0)

for word in bowA:
    wordDictA[word] += 1
for word in bowB:
    wordDictB[word] += 1
for word in bowC:
    wordDictC[word] += 1
for word in bowD:
    wordDictD[word] += 1
for word in bowE:
    wordDictE[word] += 1
for word in bowF:
    wordDictF[word] += 1

# Printing term frequency

print("SentenceA TF: ", wordDictA)
print("SentenceB TF: ", wordDictB)
print("SentenceC TF: ", wordDictC)
print("SentenceD TF: ", wordDictD)
print("SentenceE TF: ", wordDictE)
print("SentenceF TF: ", wordDictF)

print(pd.DataFrame([wordDictA, wordDictB, wordDictB, wordDictC, wordDictD, wordDictE, wordDictF]))

Output:

   CS  Guitar  Mamoosh  Piano  Sara  Student  William  a  and  is  kind  likes
0   0       0        0      2     0        0        2  0    1   0     0      2
1   0       1        0      0     1        0        0  0    0   0     0      1
2   0       1        0      0     1        0        0  0    0   0     0      1
3   0       0        1      1     0        0        0  0    0   0     0      1
4   1       0        0      0     0        1        1  1    0   1     0      0
5   0       0        0      0     1        0        0  0    0   1     1      0
6   0       0        1      0     0        0        0  0    0   1     1      0

You can simply do the following steps,

  • You can change the options for the Pandas max_columns feature as follows:

    import pandas as pd
    pd.options.display.max_columns = 10
    

    (This allows 10 columns to display, and you can change this as you need.)

  • Like that, you can change the number of rows that you need to display as follows (if you need to change maximum rows as well):

    pd.options.display.max_rows = 999
    

    (This allows to print 999 rows at a time.)

Please kindly refer to the documentation to change different options/settings for Pandas.

You can use this custom function for displaying things for a Pandas Dataframe.

def display_all(df):     # For any Dataframe df
   with pd.option_context('display.max_rows',1000): # Change number of rows accordingly
      with pd.option_context('display.max_columns',1000): # Change number of columns accordingly
          display(df)

display_all(df.head()) # Pass this function to your dataframe and voilĂ !

You don't have to use pd.set_option for the whole notebook just using for a single cell.

If you don't want to mess with your display options and you just want to see this one particular list of columns without expanding out every dataframe you view, you could try:

df.columns.values

You can also try in a loop:

for col in df.columns: 
    print(col) 
pd.options.display.max_columns = 100

You can specify the numbers of columns as per your requirement in max_columns.

The below will increase the width when NumPy arrays are printed.

It gave good results in Jupyter Notebook.

import numpy as np
np.set_printoptions(linewidth=160)

None of these answers were working for me. A couple of them would indeed print all the columns, but it would look sloppy. As in all the information was there, but it wasn't formatted correctly. I'm using a terminal inside of Neovim so I suspect that to be the reason.

This mini function does exactly what I need, just change df_data in the two places it is for your dataframe name (col_range is set to what pandas naturally shows, for me it is 5 but it could be bigger or smaller for you).

import math
col_range = 5
for _ in range(int(math.ceil(len(df_data.columns)/col_range))):
    idx1 = _*col_range
    idx2 = idx1+col_range
    print(df_data.iloc[:, idx1:idx2].describe())

It is not the answer strictly speaking, but let's remember we can df.describe().transpose() or even df.head(n).transpose(), or df.tail(n).transpose().

I also find it easier to read headers as a column when they are structured:

header1_xxx,

header2_xxx,

header3_xxx,

I think terminals and applications handle the vertical scrolling more naturally, if this is necessary after transposing.

Headers are commonly larger than their values, having all of them in one column (index) minimizes their impact on the total table width.

Finally other df descriptions can be merged as well, here is a possible idea:

def df_overview(df: pd.DataFrame, max_colwidth=25, head=3, tail=3):
    return(
        df.describe([0.5]).transpose()
        .merge(df.dtypes.rename('dtypes'), left_index=True, right_index=True)
        .merge(df.head(head).transpose(), left_index=True, right_index=True)
        .merge(df.tail(tail).transpose(), left_index=True, right_index=True)
        .to_string(max_colwidth=max_colwidth, float_format=lambda x: "{:.4G}".format(x))
    )
Related