tf.data or tf.keras.utils.Sequence. Improving efficiency of tf.data?

Viewed 701

I am trying to develop an image colorizer using autoencoders. There are 13000 training images. Each epoch takes about 45 minutes if I use tf.data and about 25 minutes if I use tf.utils.keras.Sequence. However with the use of Sequence there is a risk of deadlocks. How do I improve tf.data? I tried a couple of things but they don't seem to improve anything.

tf.data 1

image_path_list = glob.glob('datasets/imagenette/*')
data = tf.data.Dataset.list_files(image_path_list)

def tf_rgb2lab(image):
  im_shape = image.shape
  [image,] = tf.py_function(color.rgb2lab, [image], [tf.float32])
  image.set_shape(im_shape)
  return image

def preprocess(path):
  image = tf.io.read_file(path)
  image = tf.image.decode_jpeg(image, channels=3)
  image = tf.image.convert_image_dtype(image, tf.float32)
  image = tf.image.resize(image, [224, 224])
  image = tf_rgb2lab(image)
  L = image[:,:,0]/100.
  ab = image[:,:,1:]/128.
  input = tf.stack([L,L,L], axis=2)
  return input, ab

train_ds = data.repeat().map(preprocess, AUTOTUNE).batch(32).prefetch(AUTOTUNE)

tf.data 2

AUTOTUNE = tf.data.experimental.AUTOTUNE

def tf_rgb2lab(image):
  im_shape = image.shape
  [image,] = tf.py_function(color.rgb2lab, [image], [tf.float32])
  image.set_shape(im_shape)
  return image

def split_for_feed(image):  
  L = image[:,:,:,0]/100.
  ab = image[:,:,:,1:]/128.
  input = tf.stack([L,L,L], axis=-1)
  return input, ab

def read_images(path):
  image = tf.io.read_file(path)
  image = tf.image.decode_jpeg(image, channels=3)
  image = tf.image.convert_image_dtype(image, tf.float32)
  image = tf.image.resize(image, [224, 224])
  image = tf_rgb2lab(image)
  return image
  
data2 = data.repeat().map(read_images, AUTOTUNE).batch(32)
train_ds = data2.map(split_for_feed, AUTOTUNE).prefetch(AUTOTUNE)

Sequence

class ImageGenerator(tf.keras.utils.Sequence):
  def __init__(self, image_filenames, batch_size):
    self.image_filenames = image_filenames
    self.batch_size = batch_size

  def __len__(self):
    return math.ceil(len(self.image_filenames) / self.batch_size)

  def __getitem__(self, idx):
    batch = self.image_filenames[idx * self.batch_size : (idx + 1) * self.batch_size]
    X_batch = []
    y_batch = []
    for file_name in batch:
      file_name = 'datasets/imagenette/' + file_name
      try:
        color_image = transform.resize(io.imread(file_name),(224,224))
        lab_image = color.rgb2lab(color_image)
        L = lab_image[:,:,0]/100.
        ab = lab_image[:,:,1:]/128.
        X_batch.append(np.stack((L,L,L), axis=2))
        y_batch.append(ab)
      except:
        pass
    return np.array(X_batch), np.array(y_batch)
1 Answers

If your data fits in memory, try caching the preprocessing. Instead of

train_ds = data.repeat().map(preprocess, AUTOTUNE).batch(32).prefetch(AUTOTUNE)

Do

train_ds = data.map(preprocess, AUTOTUNE).batch(32).cache().repeat().prefetch(AUTOTUNE)

That way you parse each file just once, instead of repeatedly.

If you're looking to optimize the pipeline further, consider using the TF Profiler, which can tell you exactly how much time is being spent in each part of the dataset, so that you can find the bottleneck and solve it.

Related