What is the equivalent of keras NonNeg weight constraint?

user_9557861 0
Viewed 1436

Keras has an option to force the weights of the learned model to be positive:

tf.keras.constraints.NonNeg()

But I couldn't find the equivalent of this in pytorch, does anyone know how can I force my linear model's weights to be all positives?

Tried asking this on other forums but the answers were not helpful.

Let's say I have a very simple linear model as shown below, how should I change it?

class Classifier(nn.Module):

    def __init__(self,input , n_classes):
        super(Classifier, self).__init__()

        self.classify = nn.Linear( input  , n_classes)

     def forward(self, h ):

        final = self.classify(h)
        return final

I want to do exactly what the NonNeg() does but in pytorch, don't want to change what its doing.

This is the implementation of NonNeg in keras:

class NonNeg(Constraint):
    """Constrains the weights to be non-negative.
    """

    def __call__(self, w):
        w *= K.cast(K.greater_equal(w, 0.), K.floatx())
        return w
torch pytorch keras
2 Answers

You could define your own layer this way... But I am curious of why you want to do this

import torch
import torch.nn as nn

class PosLinear(nn.Module):
    def __init__(self, in_dim, out_dim):
        super(PosLinear, self).__init__()
        self.weight = nn.Parameter(torch.randn((in_dim, out_dim)))
        self.bias = nn.Parameter(torch.zeros((out_dim,)))
        
    def forward(self, x):
        return torch.matmul(x, torch.abs(self.weight)) + self.bias

Basically, it uses the absolute value of the weight.

user_53636860

Another possibility is through the exponential function:

import torch
import torch.nn as nn

class ExpLinear(nn.Module):
    def __init__(self, in_features, out_features):
        super().__init__()
        self.in_features = in_features
        self.out_features = out_features
        self.weights= nn.Parameter(torch.Tensor(out_features, in_features))
        self.bias= nn.Parameter(torch.Tensor(out_features))

    def forward(self, input):
        return nn.functional.linear(input, self.weights.exp(), self.bias.exp())

This template could also be used with the absolute value function, which I think would be more pythonic than the accepted answer (from @Serge de Gosson de Varennes).

user_98157080
Related