Teacher force training PyTorch

user_6997665 0
Viewed 243

I am trying to do a seq2seq prediction. For this, I have a LSTM layer followed by a fully connected layer. I employ Teacher training during the training phase and would like to skip this (I maybe wrong here) during testing phase. I have not found a direct way of doing this so I have taken the approach shown below.

def forward(self, inputs, future=0, teacher_force_ratio=0.2, target=None):
    outputs = []
    for idx in range(future):
        rnn_out, _ = self.rnn(inputs)
        output = self.fc1(rnn_out)
        if self.teacher_training:
            new_input = output if np.random.random() >= teacher_force_ratio else target[idx]
        else:
            new_input = output
        inputs = new_input

I use a bool variable teacher_training to check if Teacher training is needed or not. Is this correct? If yes, is there a better way to do this? Thanks.

pytorch deep-learning machine-learning python-3.x
1 Answers

In PyTorch all classes that extend nn.Module have a kwarg boolean param called training . So instead of teacher_training we should simply use training param. This param is automatically set depending on your model training mode (model.train() and model.eval()).

user_82485230
Related