Is the difference between autoregressive transformer and non-autoregressive transformer is only the attention masks in decoder?

Viewed 281

The way i think is that,

The attention mask (not padding mask) for decoder of an autoregressive transformer model is a subsequent generated mask like as

[0, 1, 1, 1, 1]
[0, 0, 1, 1, 1]
[0, 0, 0, 1, 1]
[0, 0, 0, 0, 1]

While there's no attention mask being fed to the decoder of non-autoregressive transformer model, there's only a padding mask.

Am i correct?

0 Answers
Related