I am new to deep learning, i want to build a model that can identify similar images, i am reading classification is a Strong Baseline for Deep Metric Learning research paper. and here is they used the phrase: "remove the bias term inthe last linear layer". i have no idea what is bias term is and how to remove it from googlenet or other pretrained models. if someone help me out with this it would be great! :)
What is bias node in googlenet and how to remove it?
Viewed 40
1 Answers
To compute the layer n outputs, a linear neural network computes a linear combination of the layer n-1 output for each layer n output, adds a scalar constant value to each layer n output (the bias term), and then applies an activation function. In pytorch, one could disable the bias in a linear layer using:
layer = torch.nn.Linear(n_in_features, n_out_features, bias=False)
To overwrite the existing structure of, say, the googlenet included in torchvision.models defined here, you can simply override the last fully connected layer after initialization:
from torchvision.models import googlenet
num_classes = 1000 # or whatever you need it to be
model = googlenet(num_classes)
model.fc = torch.nn.Linear(1000,num_classes,bias = False)
Related