sampler argument in DataLoader of Pytorch

Question

While using Pytorch's DataLoader utility, in sampler what is the purpose of RandomIdentitySampler? And in RandomIdentitySampler there is an argument instances. Does instances depends upon number of workers? If there is are 4 workers then should there be 4 instances as well?

Following is the chunk of code:

c_dataloaders = DataLoader(Preprocessor(cluster_dataset.train_set,
                                        root=cluster_dataset.images_dir,
                                        transform=train_transformer),
                                        batch_size=args.batch_size_stage2,
                                        num_workers=args.workers,
                                        sampler=RandomIdentitySampler(cluster_dataset.train_set,
                                        args.batch_size_stage2,
                                        args.instances)

user_4228275 · Answer

This sampler is not part of the PyTorch or any other official lib (torchvision, torchtext, etc.). Anyway, there is a RandomIdentitySampler in the torchreid from KaiyangZhou. Assuming this is the case:

While using Pytorch's DataLoader utility, in sampler what is the purpose of RandomIdentitySampler?

As you can see in the DataLoader documentation: the sampler "defines the strategy to draw samples from the dataset". More specifically, based on RandomIdentitySampler documentation, it "randomly samples N identities each with K instances".

And in RandomIdentitySampler there is an argument instances. Does instances depends upon number of workers?

Based on the previous answer, you can note that instances does not depend on the number of workers. It simply sets the number of instances of each identity that will be drawn from the dataset for each batch.

If there are 4 workers then should there be 4 instances as well?

Not necessarily. The only constraint is that the number of instances should not be smaller than the batch size.

sampler argument in DataLoader of Pytorch

1 Answers