How to use torchvision.datasets.ImageFolder on a folder stored in AWS S3 bucket

Viewed 384

Lets say I have a huge image data stored in AWS S3 bucket. I need to use trochvision.datasets.ImageFolder as my training data is in appropriate folders. So I want to use the following code in my Sagemaker notebook.

from torch.utils.data import DataLoader
from torchvision import datasets, transforms

data_location = "s3_bucket/folder" 
batch_size= 16

transforms_= transforms.Compose([ transforms.RandomHorizontalFlip(p= 0.25),
                                transforms.RandomRotation(30),
                               transforms.RandomCrop(256, pad_if_needed=True),
                               transforms.ToTensor(),
                               transforms.Normalize([0.5, 0.5,0.5], [0.5,0.5,0.5]),
                               ])

train_dataset= datasets.ImageFolder(data_location, transform= transforms_)
train_loader= DataLoader(train_dataset, batch_size= batch_size, shuffle= True)

How can I do this?

0 Answers
Related