tf.keras.preprocessing.image.ImageDataGenerator is super simple and easy to use to perform data augmentation. However, it seems to be much slower than using tf.data.Dataset to load data. I tried to load images tf.data.Dataset but couldn't figure out how to do the same data augmentation such as randomly shifting the width and height, randomly rotate, etc. I see in github that all these tf.keras.image.ImageDataGenerator augmentations seem to be using this function tf.keras.preprocessing.image.apply_affine_transform; however, the input must be numpy arrays.

For the best performance, do I have to rewrite the tf.keras.preprocessing.image.apply_affine_transform function so it takes TensorFlow tensors or can I just change the image to numpy during preprocessing phase in tf.data.Dataset, use the apply_affine_transform function for data augmentation, and cast it back to a Tensor?

Additionally, there is also tf.data.Dataset.from_generator() which looks like it can take an ImageDataGenerator.

Which is faster and more efficient for data loading: option 1, 2, or 3?

import tensorflow as tf

AUTOTUNE = tf.data.experimental.AUTOTUNE

batch_size=32

option 1

def preprocess(filename): tfImg = tf.io.read_file(file_path) numpyImg = tf.image.decode_jpeg(tfImg, channels=3).numpy() augNumpy = tf.keras.preprocessing.image.apply_affine_transform(numpyImg, change_some_arguments_here_for_augmentation) return tf.cast(augNumpy, tf.float32). # return augmented image as Tensor

list_ds = tf.data.Dataset.list_files(str(data_dir/'*/*'))

newAugmentedDataset = list_ds.map(preprocess, num_parallel_calls=AUTOTUNE).shuffle(buffer_size=1000).repeat().batch_size(batch_size).prefetch(AUTOTUNE)

option 2

def preprocess(filename): tfImg = tf.io.read_file(file_path) tfImg = tf.image.decode_jpeg(tfImg, channels=3) return rewritten_apply_affine_function(tfImg,...)

list_ds = tf.data.Dataset.list_files(str(data_dir/'*/*'))

newAugmentedDataset = list_ds.map(preprocess, num_parallel_calls=AUTOTUNE).shuffle(buffer_size=1000).repeat().batch_size(batch_size).prefecth(AUTOTUNE)

option 3

train_data_generator = ImageDataGenerator(augmentations_here)

ftrain_generator = train_data_generator.flow_from_directory(directory_here, shuffle=False)

ftrain_generator_ds = tf.data.Dataset.from_generator(lambda : ftrain_generator, output_types=(tf.float32), output_shapes = (tf.TensorShape([None, height_here, width_here, num_channel]).prefetch(AUTOTUNE)

any of the options above goes into .fit

model.fit(dataset_here, steps_per_epoch=sample_count/batch_size)