python keras: Image reading and loading

user_12354419 0
Viewed 105

I want to read images from multiple directories. I used Keras Image datagenerator for this. After applying a preprocessing function, now I want to load entire data in a single list. I tried doing the following code. 

def preProcess(X):
    X = X.astype('float32')
    X = (X - 127.5) / 127.5
    return X

batch_sz=32
path = "/content/drive/My Drive/new_net/mini_unet_data/labeled/"
test_datagen = ImageDataGenerator(preprocessing_function=preProcess)
test_gen = test_datagen.flow_from_directory(directory=path,target_size=(256,256),batch_size=32,color_mode="rgb",class_mode="sparse",shuffle=True,seed=42)
test_images=[]
test_labels=[]


for i in range(int(test_gen.n/batch_sz)):
    tmp1, tmp2 = test_gen.next()
    test_images.append(tmp1)
    test_labels.append(tmp2)
test_images = np.asarray(test_images)
test_labels = np.asarray(test_labels)

test_images = np.reshape(test_images,(test_images.shape[0]*test_images.shape[1],test_images.shape[2],test_images.shape[3],test_images.shape[4]))
test_labels = np.squeeze(np.reshape(test_labels,(test_labels.shape[1]*test_labels.shape[0])))

The above code works without error. But it takes time to load around 1500 images (more than 10 minutes). So is there better way to achieve this faster? I tried using glob and open cv commands. That is too slow as well. Thank You.

keras numpy python
1 Answers

You can try two things to speed up your process each time you run the evaluation code.

  1. Convert the images to .npy format. Read it using cv2 and then change it from BGR to RGB and save it.
  2. You can generate the npy array of whole test set and then use predict/evaluate generator.

(Can't comment)

user_72122430
Related