Huggingeface model generator method do_sample parameter

user_4281353 0
Viewed 664

What does do_sample parameter of the generate method of the Hugging face model do?

Generates sequences for models with a language modeling head. The method currently supports greedy decoding, multinomial sampling, beam-search decoding, and beam-search multinomial sampling.

do_sample (bool, optional, defaults to False) – Whether or not to use sampling; 
use greedy decoding otherwise.

When the Beam search length is 1, it can be called greedy. Does do_sample=False mean the same?

huggingface-transformers
1 Answers

If your do_sample=True, your generate method will use Sample Decoding. You can look at the different decoding strategies here. You can also play with the temperature parameter.

Now, if do_sample=False and num_beams=1, then your generate method will use greedy decoding.

user_76822590
Related