I am trying to write a preprocessing function using OpenCV through tensorflow dataset pipeline. Following this post do not work in my case.
To explicit my point, consider this dummy tensor:
import tensorflow as tf
import numpy as np
ds1 = tf.random.uniform(
(6,5,4,3),
minval=0,
maxval=None,
dtype=tf.dtypes.float64,
seed=None,
name=None
)
ds2 = tf.data.Dataset.from_tensor_slices(ds1).batch(batch_size=2)
ds2
Out[4]: <BatchDataset element_spec=TensorSpec(shape=(None, 5, 4, 3), dtype=tf.float64, name=None)>
Next, my goal is to apply preprocessing step on these "array" (a.k.a images obtained using tf.keras.preprocessing.image_dataset_from_directory in practices...)
Some dummy functions :
def preprocess_images(x):
return x+1
def parse_func_decorator(x):
return tf.py_function(preprocess_images, [x], tf.float64)
Now begins the mystery that I want to understand: Applying the preprocessing function via py_function gives unknown shape :
ds3 = ds2.map(parse_func_decorator)
ds3
Out[7]: <MapDataset element_spec=TensorSpec(shape=<unknown>, dtype=tf.float64, name=None)>
On the other hand, mapping the preprocessing function, directly, conserve dimension
ds5 = ds2.map(preprocess_images)
ds5
Out[9]: <MapDataset element_spec=TensorSpec(shape=(None, 5, 4, 3), dtype=tf.float64, name=None)>
What am I missing ?