TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType' Deep Learning

user_14047351 0
Viewed 974

We would like to split our data into 3 parts however we get the following error:

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Below you can see our python-code:

data = pd.read_csv('/content/drive/My Drive/GTruth.csv')

def dataset_gen(data, size = (224,224,3)):
    
    img_data = []
    labels = []
    directory='/content/drive/My Drive/Lungenbilder_gekürzt'
    for filename in os.listdir(directory) :
     if filename.endswith(".jpeg") or filename.endswith(".jpg"):
     #for img_name, pollen_carrying in zip(data['Id'], data['Ground_Truth']):
        img = io.imread(os.path.join(str(directory), str(filename)))
        img = transform.resize(img, size, mode = 'constant')
        img_data.append(img)
        labels.append(kv_dict.get((filename.split('.')[0])))
     else:
         continue    
    return np.array(img_data), np.array(labels)

x, y = dataset_gen(data)

#Train Test Split
X_train, X_1, y_train, y_1 = train_test_split(x, y, test_size=0.5, random_state=42)

y_train = tf.keras.utils.to_categorical(y_train, num_classes = 2) #erstellt Matrix
y_1 = tf.keras.utils.to_categorical(y_1, num_classes = 2)

#Train Test Split
X_cv, X_test, y_cv, y_test = train_test_split(X_1, y_1, test_size=0.2, random_state=42)

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-116-474212bb9856> in <module>()
      2 X_train, X_1, y_train, y_1 = train_test_split(x, y, test_size=0.5, random_state=42)
      3 
----> 4 y_train = tf.keras.utils.to_categorical(y_train, num_classes = 2) #erstellt Matrix
      5 y_1 = tf.keras.utils.to_categorical(y_1, num_classes = 2)
      6 

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/utils/np_utils.py in to_categorical(y, num_classes, dtype)
     67 
     68   """
---> 69   y = np.array(y, dtype='int')
     70   input_shape = y.shape
     71   if input_shape and input_shape[-1] == 1 and len(input_shape) > 1:

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

Thanks for your help!

Sandra and Leonie

google-colaboratory deep-learning python
1 Answers

I think the error is caused by your y_train dataset containing NoneType values.

For example here I replicate your error using a simple numpy.array

import numpy as np   
tst = np.array([1, '2', None, 3]) 
np.array(tst, dtype='int') 

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-5-37b79816a629> in <module>
----> 1 np.array(tst, dtype='int')

TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'

A good pre-processing step when doing ML is to drop rows with NaN's (particularly in your y variable) after you read in your data (or you can fill them if you know what the default value should be)

data = pd.read_csv('/content/drive/My Drive/GTruth.csv')
data.dropna(inplace=True)

When you do it for x and y make sure you drop the same indices otherwise your features (x) will misalign with your labels (y). So here I only keep the features x which have labels y that are not None

x = x[y!=None]
y = y[y!=None]
user_59969720
Related