I'm studying neural networks and I have seen that each time I run my code, the obtained accuracy after the training changes. I think that it depends by the initialization of the weights, I don't know if other stuff affect it. Additionally, I would like to compare two networks, so in order to do a right comparison I would like to do something in order to get the same performance each time I run a model. Has anyone a suggestion? I'm using pytorch, and I tried to add the following code in the train.py file, before the definition of the network
seed = 3
torch.manual_seed(seed)
torch.backends.cudnn.deterministic = True
torch.backends.cudnn.benchmark = False
Furthermore, in the UNET network (it is composed of convolutional layers, max-pooling layers and batch normalization layers), I added the following row before each convolutional layer:
torch.manual_seed(3)
However, each time I train the model, the obtained performance changes.