selecting columns for scatter plot from csv

Question

I am trying to make a scatter matrix from a large csv and the scatter matrix works fine on a smaller file, but when applying it to my larger file I can't get the correct colum output, it just returns one of 2 scatter plots and when I try increasing the number of where to start selecting from it throws an error on output:

    Traceback (most recent call last):
  File "C:\Users\Kinkerman\Downloads\Uni\Data Analytics\Python\Python files wk 5\scatter_matrix2.py", line 8, in 
    scatter_matrix(data.loc[:, "V5":"V8"], diagonal="kde")
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\plotting\_misc.py", line 128, in scatter_matrix
    return plot_backend.scatter_matrix(
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\plotting\_matplotlib\misc.py", line 50, in scatter_matrix
    fig, axes = create_subplots(naxes=naxes, figsize=figsize, ax=ax, squeeze=False)
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\pandas\plotting\_matplotlib	ools.py", line 267, in create_subplots
    ax0 = fig.add_subplot(nrows, ncols, 1, **subplot_kw)
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\matplotlib\figure.py", line 772, in add_subplot
    ax = subplot_class_factory(projection_class)(self, *args, **pkw)
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\matplotlib\axes\_subplots.py", line 36, in __init__
    self.set_subplotspec(SubplotSpec._from_subplot_args(fig, args))
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\matplotlib\gridspec.py", line 597, in _from_subplot_args
    gs = GridSpec._check_gridspec_exists(figure, rows, cols)
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\matplotlib\gridspec.py", line 225, in _check_gridspec_exists
    return GridSpec(nrows, ncols, figure=figure)
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\matplotlib\gridspec.py", line 385, in __init__
    super().__init__(nrows, ncols,
  File "C:\Users\Kinkerman\AppData\Local\Programs\Python\Python310\lib\site-packages\matplotlib\gridspec.py", line 52, in __init__
    raise ValueError(
    ValueError: Number of columns must be a positive integer, not 0ValueError: Number of columns must be a positive integer, not 0

This is my code so far:

    import pandas as pd
    import matplotlib.pyplot as plt
    from pandas.plotting import scatter_matrix

    #read from file and rename columns "Vi"
    data = pd.read_csv('principal_offence_category_april_2014.csv')
    data.columns = ["V"+str(i) for i in range(1, len(data.columns)+1)]

    #select columns and plot in the scatter matrix
    scatter_matrix(data.loc[:, "V5":"V8"], diagonal="kde")
    plt.tight_layout()
    plt.show()

this is the file I am using >>> https://drive.google.com/file/d/1X6PN_EtMpspazVxI6d3h3J08NWQIb11Y/view?usp=sharing

user_4985733 · Answer

You can use the thousands parameter to specify that , is used and na_values to convert - into NaN values. You then need an approach to convert the % entries into floats. For example:

data['V5'] = data['V5'].str.rstrip('%').astype('float')

This could be applied to some of the percentage columns:

import pandas as pd
import matplotlib.pyplot as plt
from pandas.plotting import scatter_matrix

#read from file and rename columns "Vi"
data = pd.read_csv('principal_offence_category_april_2014.csv', thousands=',', na_values=['-'])
data.columns = [f"V{i}" for i in range(1, len(data.columns)+1)]
cols = [f'V{col}' for col in range(3, 12, 2)]       # e.g. percentage columns  V3, V5, V7, V9, V11

for col in cols:
    data[col] = data[col].str.rstrip('%').astype('float')

#select columns and plot in the scatter matrix
scatter_matrix(data[cols], diagonal="kde")
plt.tight_layout()
plt.show()

It would give this output for your file:

selecting columns for scatter plot from csv

1 Answers