Keras custom data generator is very slow

I am training an autoencoder with Keras custom data generator. Data is big enough to not fit into the memory.

Generator:

class Mygenerator(Sequence):
    def __init__(self, x_set, y_set, batch_size):
        self.x, self.y = x_set, y_set
        self.batch_size = batch_size

    def __len__(self):
        return int(np.ceil(len(self.x) / float(self.batch_size)))

    def __getitem__(self, idx):
        batch_x = self.x[idx * self.batch_size:(idx + 1) * self.batch_size]
        batch_y = self.y[idx * self.batch_size:(idx + 1) * self.batch_size]

        # read your data here using the batch lists, batch_x and batch_y
        x = [np.reshape(np.load(filename),(52,52,1)) for filename in batch_x] # load array and reshape to fit input layer
        y = [np.reshape(np.load(filename),(52,52,1)) for filename in batch_y] # load array and reshape to fit input layer
        return np.array(x), np.array(y)

Model fit_generator:

XTRAINFILES = glob.glob("C:\\x_train\\*.npy") 
YTRAINFILES = glob.glob("C:\\y_train\\*.npy")
XTESTFILES = glob.glob("C:\\x_test\\*.npy") 
YTESTFILES = glob.glob("C:\\y_test\\*.npy") 

autoencoder_model.fit_generator(Mygenerator(XTRAINFILES, YTRAINFILES, 128),
                    epochs=EPOCHES, workers=8, steps_per_epoch=ceil( len(XTRAINFILES) / 128)
                    validation_data=Mygenerator(XTESTFILES, YTESTFILES, 128),
                    validation_steps=ceil( len(XTESTFILES) / 128),
                    callbacks=[tb_output, checkpoint_callback])

Keras gives an ETA between 2 and 3 hours. testing without the custom generator with just a little bit less data to fit in the memory had ETA of 20 to 30 mins per epoch.

Insights about PC specs:

• GPU: Geforce RTX 2080 Ti
• Ram: 128 GB

Attempted solution: adding workers = 8 to the fit generator, improved the time a little but still not close enough to the expected