How to read a portion of the Azure File Dataset(containing 1000 Images) in different train nodes.
I want a sampling where all the images are covered.
I am looking for an option like tensorflow.dataset.shard().
Thanks.
Azure Machine Learning FileDataSet Images - Shard/Split equally to nodes
Viewed 184
1 Answers
You can use the the ParallelRunStep in Pipelines with FileDataset as input. See: https://docs.microsoft.com/en-us/azure/machine-learning/how-to-use-parallel-run-step and https://docs.microsoft.com/en-us/python/api/azureml-contrib-pipeline-steps/azureml.contrib.pipeline.steps.parallelrunstep?view=azure-ml-py
Related