how to create a PrefetchDataset whit the images from two directories?

Viewed 625

hi im trying to follow this tutorial

https://www.tensorflow.org/tutorials/images/segmentation#define_the_model

but using my own data(which is not working very well), suppose i have the path to the mask(target_dir) and the original(traind_dir) images, how can i make a PrefetchDataset like this:

[1]:train_dataset.__dict__
[1]: {'_input_dataset': <RepeatDataset shapes: ((None, 128, 128, 3), (None, 128, 128, 1)), types: (tf.float32, tf.float32)>,
 '_buffer_size': <tf.Tensor: shape=(), dtype=int64, numpy=-1>,
 '_variant_tensor_attr': <tf.Tensor: shape=(), dtype=variant, numpy=<unprintable>>,
 '_self_setattr_tracking': True,
 '_self_unconditional_checkpoint_dependencies': [TrackableReference(name='_variant_tracker', ref=<tensorflow.python.data.ops.dataset_ops._VariantTracker object at 0x7f36fc7db320>)],
 '_self_unconditional_dependency_names': {'_variant_tracker': <tensorflow.python.data.ops.dataset_ops._VariantTracker at 0x7f36fc7db320>},
 '_self_unconditional_deferred_dependencies': {},
 '_self_update_uid': -1,
 '_self_name_based_restores': set(),
 '_self_saveable_object_factories': {},
 '_variant_tracker': <tensorflow.python.data.ops.dataset_ops._VariantTracker at 0x7f36fc7db320>,
 '_graph_attr': <tensorflow.python.framework.ops.Graph at 0x7f37279a32e8>,
 '_options_attr': <tensorflow.python.data.ops.dataset_ops.Options at 0x7f36fc7db710>}

whit my own .jpg files stored in traind_dir and target_dir?

the train_dataset up there, is obtained from the oxford_iiit_pet dataset and following the tensorflow.org tutorial

edit: in my tries i got whit this, if i use

getattr(dataset["train"],"_input_dataset")

i get

<ParallelMapDataset shapes: {file_name: (), image: (None, None, 3), label: (), segmentation_mask: (None, None, 1), species: ()}, types: {file_name: tf.string, image: tf.uint8, label: tf.int64, segmentation_mask: tf.uint8, species: tf.int64}>

so i googled for ParallelMapDataset, i found that i can create this dataset with

tf.raw_ops.ParallelMapDataset(
    input_dataset, other_arguments, num_parallel_calls, f, output_types, output_shapes
)

but i have some questions

what is supose to be in other_arguments? the tf.float32 list?

to pairing the input and the target images what f and num_parallel_calls i should use?

0 Answers
Related