Im migrating from Keras's ImageDataGenerators to tf.data and I have issues with reproducing the same augmentation I used to have in generator. While there are regularization layers for Random Zoom, Random Rotation or Random Flip in tf.keras, there is none for Shear transformation.
I am trying to write a custom layer using tf.keras.preprocessing.image.random_shear. I know that some image-related functions do not work with batches (random_shear included), so I use the map_fn:
class Shear(tf.keras.layers.Layer):
'''
Define tf.keras.layer which randomly shears the input images.
The layer is active only during inference
'''
def __init__(self, factor = 30, **kwargs):
'''
Initialize the Shear layer
param factor: intensity of transformation in degrees
'''
super().__init__(**kwargs)
self.factor = factor
def shear(self, image):
'''
Randomly shear a single image
'''
return tf.keras.preprocessing.image.random_shear(image, self.factor, 0, 1, 2) #channels last
def call(self, x, training = None):
'''
Randomly shear a batch of images
'''
if not training:
return x
return tf.map_fn(self.shear, x)
The layer itself works: I transformed and visualized some batches with it, and the result is as expected. However, when I connect this layer to model, during training on GPU I get this error:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:855 train_function *
return step_function(self, iterator)
train.py:72 shear *
return tf.keras.preprocessing.image.random_shear(image, self.factor, 0, 1, 2)
/usr/local/lib/python3.6/dist-packages/keras_preprocessing/image/affine_transformations.py:115 random_shear *
x = apply_affine_transform(x, shear=shear, channel_axis=channel_axis,
/usr/local/lib/python3.6/dist-packages/keras_preprocessing/image/affine_transformations.py:327 apply_affine_transform *
channel_images = [ndimage.interpolation.affine_transform(
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:520 __iter__
self._disallow_iteration()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:513 _disallow_iteration
self._disallow_when_autograph_enabled("iterating over `tf.Tensor`")
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:491 _disallow_when_autograph_enabled
" indicate you are trying to use an unsupported feature.".format(task))
OperatorNotAllowedInGraphError: iterating over `tf.Tensor` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.
I tried to enable numpy behaviour, but it didn't help. I tried to apply shear transform inside tf.data pipeline, but for some reason I can't map shear function on batches + I'd like to have all preprocessing inside the model as layers. I'd use shear_x/shear_y functions from tfa.image, but they don't have proper filling modes.
I created similar layers with tf.image functions, such as random_jpeg_quality, and they work just fine, but when I use something outside of tf.image, I get errors :(
Will be thankful for any suggestions about resolving this issue
P. S. I use TF 2.5.0