I'm currently working on implementing a neural CRF as a project for school and am looking around for repos to reference.
I encountered this one the other day and have been completely stumped by the implementation of the forward algorithm.
T = feats.shape[1]
batch_size = feats.shape[0]
# alpha_recursion,forward, alpha(zt)=p(zt,bar_x_1:t)
log_alpha = torch.Tensor(batch_size, 1, self.num_labels).fill_(-10000.).to(self.device)
# normal_alpha_0 : alpha[0]=Ot[0]*self.PIs
# self.start_label has all of the score. it is log,0 is p=1
log_alpha[:, 0, self.start_label_id] = 0
# feats: sentances -> word embedding -> lstm -> MLP -> feats
# feats is the probability of emission, feat.shape=(1,tag_size)
for t in range(1, T):
log_alpha = (log_sum_exp_batch(self.transitions + log_alpha, axis=-1) + feats[:, t]).unsqueeze(1)
# log_prob of all barX
log_prob_all_barX = log_sum_exp_batch(log_alpha)
return log_prob_all_barX
From my understanding, the forward algorithm and its log alpha should be the log of the following where s_m is the scoring function - transition score from previous to current tag + emission score of neural hidden state/feature:
it seems to me that the code should be something more akin to log_alpha = log_sum_exp(transition_score + feat_score + log_alpha) if the log is applied.