How to find the best value for mean and STD of Normalize in torchvision.transforms

user_2634046 0
Viewed 1552

I have started working with PyTorch and cannot figure it how I am supposed to find mean and std as the input parameters of normalise.

I have seen this

transforms.Normalize((0.485, 0.456, 0.406), (0.229, 0.224, 0.225)) #https://pytorch.org/vision/stable/transforms.html#

and in another example:

transformation = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5,0.5,0.5],std=[0.5,0.5,0.5])
])  #https://github.com/MicrosoftDocs/ml-basics/blob/master/challenges/05%20%20-%20Safari%20CNN%20Solution%20(PyTorch).ipynb

so, how am I supposed to know or get these values if I have a set of images? Are these three parameters are related to R G B also?

pytorch python
2 Answers

Like I mentioned in the comment, you can use ImageNet statistics for general domains. Otherwise, you can use this from PyTorch forms here to calculate mean and std for your own dataset which needs to be normalised.

class MyDataset(Dataset):
    def __init__(self):
        self.data = torch.randn(100, 3, 24, 24)
        
    def __getitem__(self, index):
        x = self.data[index]
        return x

    def __len__(self):
        return len(self.data)
    

dataset = MyDataset()
loader = DataLoader(
    dataset,
    batch_size=10,
    num_workers=1,
    shuffle=False
)


mean = 0.
std = 0.
nb_samples = 0.
for data in loader:
    batch_samples = data.size(0)
    data = data.view(batch_samples, data.size(1), -1)
    mean += data.mean(2).sum(0)
    std += data.std(2).sum(0)
    nb_samples += batch_samples

mean /= nb_samples
std /= nb_samples
user_129275190

suppose you have already X_train which is a list of numpy matrix eg. 32x32x3:

X_train = X_train / 255 #normalization of pixels
train_mean = X_train.reshape(3,-1).mean(axis=1)
train_std = X_train.reshape(3,-1).std(axis=1)

then you can pass this last 2 variables in your normalizer transformer :

transforms.Normalize(mean = train_mean ,std = train_std)
user_36881700
Related