How to Implement Scheduled Sampling in Keras?

Question

How can I implement the concept of Scheduled Sampling while training an LSTM Encoder-Decoder with teacher forcing in Keras?

The source paper is here: https://arxiv.org/abs/1506.03099

From this post, I see that pure Tensor-flow supports this training helper:

scheduled sampling in Tensorflow

https://www.tensorflow.org/api_docs/python/tf/contrib/seq2seq/ScheduledOutputTrainingHelper

Is there a way to do this in Keras, maybe using a custom backend function?

user_14290244 · Answer

Tensorflow contrib API is deprecated in Tensorflow2.x version. Few libraries of tf.contrib are moved to Tensorflow addons.

Use tfa.seq2seq.ScheduledOutputTrainingSampler

tfa.seq2seq.ScheduledOutputTrainingSampler(
    sampling_probability: tfa.types.TensorLike,
    time_major: bool = False,
    seed: Optional[int] = None,
    next_inputs_fn: Optional[Callable] = None
)

For more information on the library find here

How to Implement Scheduled Sampling in Keras?

1 Answers