on the fly generation with Dataset api tensorflow

user_2426955 0
Viewed 1156

I have a function which produces feature and target tensors. E.g. 

x,t = myfunc() ##x,t tensors

How can I integrate this with TensorFlow's dataset API for continuous training? Ideally I would like to use dataset to set things like batch, transformations.

Edit for clarification: The problem being I would like to not just put x and t in my graph but make a dataset from them so that I can use the same dataset processing that I have implemented for (normal) finite datasets I can load into memory and feed into the same graph using an initializable iterator.

tensorflow-datasets tensorflow python
2 Answers

If x and t are tensors, you can create a dataset by calling tf.data.Dataset.from_tensors or tf.data.Dataset.from_tensor_slices (documentation here).

The difference between them is that from_tensors combines the input tensors into a single element in the dataset. from_tensor_slices creates a dataset with one element for each slice.

user_9349040
Related