Accuracy for every epoch in Pytorch

user_13319580 0
Viewed 5241

I'm struggling to calculate accuracy for every epoch in my training function for CNN classifier in Pytorch.

After I run this script, it always prints out 0, 0.25 or 0.75 which is obviously wrong. I'm guessing the problem are the inputs of the get_accuracy function (outputs and labels) as they are not accumulated for the entire epoch but not sure how to fix that.

Ideally, I'd like to print out both train and test accuracy for every epoch.

def get_accuracy(pred, actual):
  assert len(pred) == len(actual)

  total = len(actual)
  _, predicted = torch.max(pred.data, 1)
  correct = (predicted == actual).sum().item()
  return correct / total


def train_model(model):
    for epoch in range(epochs):
        running_loss = 0.00
        for i, data in enumerate(trainloader, 0):
            inputs, labels = data
            inputs, labels = inputs.to(device), labels.to(device)

            optimizer.zero_grad()

            outputs = cnn(inputs)

            loss = criterion(outputs, labels) 
            loss.backward() 
            optimizer.step() 

            running_loss += loss.item() #calculates loss of the batch in average

        running_loss /= len(trainloader)

        training_accuracy = get_accuracy(outputs, labels)
        test_accuracy = 'todo'

        print('='*10,'iteration: {0}'.format(epoch+1),'='*10,)
        print('\n Loss: {0} \n Training accuracy:{1}% \n Test accuracy: {2}%'.format(running_loss, training_accuracy, test_accuracy))

    print('Finished Training')
conv-neural-network pytorch deep-learning neural-network
2 Answers

I assume you want to calculate accuracy for multiclass case (so your classes are of form [0, 1, 2, 3, ..., N]).

You are using maximum while it should be argmax, for example:

def accuracy(predictions, labels):
    classes = torch.argmax(predictions, dim=1)
    return torch.mean((classes == labels).float())

This return index with maximum value, while you are returning maximum probability (or unnormalized probability). As probability is almost never equal to 1 you should have 0 accuracy always (up to float precision, it might sometimes correctly be really close to 0 or 1 so it may "hit").

For example 0.9 != 2 and you can never predict class 2, but you might, by accident, predict class 1 (0.999999999 ~= 1).

This function should be called within your inner loop, just like you calculate loss, so it would be:

for epoch in range(epochs):
    running_loss = 0.00
    running_accuracy = 0.00
    for i, data in enumerate(trainloader, 0):
        inputs, labels = data inputs, labels = inputs.to(device), labels.to(device)
        optimizer.zero_grad() 
        outputs = cnn(inputs) 
        loss = criterion(outputs, labels)
        loss.backward() 
        optimizer.step() 
        running_loss += loss.item()
        running_accuracy += accuracy(outputs, labels)
    running_loss /= len(trainloader)
    running_accuracy /= len(trainloader)

Same thing would go for test and validation, just switch your model to evaluate mode (via model.eval()) and disable gradient using with torch.no_grad(): context manager).

user_108864200

Well, Firstly your inputs and outputs do not not accumulated for the entire epoch, while calculating accuracy only the last iteration of your trainloader is considered If you want to calculate accuracy then you have 2 options,

  1. Either accumulate input and output manually after each iteration and then calculate accuracy.
  2. Calculate accuracy in each iteration and average them over one epoch before printing.
user_113269440
Related