Pandas df.isna().sum() not showing all column names

user_9715720 0
Viewed 1877

I have simple code in databricks:

import pandas as pd

data_frame = pd.read_csv('/dbfs/some_very_large_file.csv')
data_frame.isna().sum()

Out[41]:
A    0
B    0
C    0
D    0
E    0
     ..
T     0
V     0
X     0
Z     0
Y     0
Length: 287, dtype: int64

How can i see all column (A to Y) names along with is N/A values? Tried setting pd.set_option('display.max_rows', 287) and pd.set_option('display.max_columns', 287) but this doesn't seem to work here. Also isna() and sum() methods do not have any arguments that would allow me to manipulate output as far as i can say.

pandas python
1 Answers

The default settings for pandas display options are set to 10 rows maximum. If the df to be displayed exceeds this number, it will be centrally truncated. To view the entire frame, you need to change the display options.

To display all rows of df:

pd.set_option('display.max_rows',None)

Ex:

>>> df
     A   B   C
0    4   8   8
1   13  17  13
2   19  13   2
3    9   9  16
4   14  19  19
..  ..  ..  ..
7    7   2   2
8    5   7   2
9   18  12  17
10  10   5  11
11   5   3  18

[12 rows x 3 columns]
>>> pd.set_option('display.max_rows',None)
>>> df
     A   B   C
0    4   8   8
1   13  17  13
2   19  13   2
3    9   9  16
4   14  19  19
5    3  17  12
6    9  13  17
7    7   2   2
8    5   7   2
9   18  12  17
10  10   5  11
11   5   3  18

Documentation: pandas.set_option

user_151608930
Related