I set 'truncation=True' when using Tapas Tokenizer in transformers, but still get a warning of not setting truncation explicitly

Viewed 27

The warning is:

Truncation was not explicitly activated but max_length is provided a specific value, please use truncation=True to explicitly truncate examples to max length. Defaulting to 'longest_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you can select this strategy more precisely by providing a specific strategy to truncation.

and my code is

tokenizer = AutoTokenizer.from_pretrained("google/tapas-base")
item = tokenizer(table = table, queries = queries, max_length = 256, truncation=True)

The version of my transformers 4.15.0. I really wonder what the problem is.

0 Answers
Related