Google Colab Session keeps crushing

Viewed 27

I am working on a CNN in Google Colab. I have 25 GB of RAM provided by Colab, yet my sheet keeps crushing when I want to load in my input data. My input data are 1800 pictures of size 1080x1920x3. From other questions in this forum I have the suspicion that my code uses too much memory. However, I consider my code already as pretty neat.

Do you know why my Colab keeps crushing and how I can avoid this error?

X = []
y = []

classes = ['asphalt_track_forest', 'asphalt_track_field'] 
base_path = '/data/img/' 


for i, target in enumerate(classes):
    files = os.listdir(base_path+target)

    for file in files:
        # load the image
        img = load_img(base_path+target+'/'+file)
        img = np.int32(img)

        # convert it to an array
        img_array = img_to_array(img)
       

        # append the array to X
        X.append(img_array)

        # append the numeric target to y
        y.append(i)

X = np.array(X)
y = np.array(y)

shuffler = np.random.permutation(len(X))
X = X[shuffler]
y = y[shuffler]

print(f'X input:{X.shape}')
print(f'y input:{y.shape}')
0 Answers
Related