GPT-3 long input posts for Question Answering

user_11132563 0
Viewed 524

From my understanding, GPT-3 is "trained" for a specific task by including some labelled examples before the desired/test example. In Question Answering, this includes a context and a question. In this situation, the input prompt can become long. How do people address this?

I am using the Hugging Face GPT-J implementation, and there is an input token limit (of 2000). However, when including multiple qa examples in the prompt (especially with the contexts), it quickly reaches this limit, limitting the amount of example prompts to be inputted. Does anyone know how this issue is handled in a GPT-J setting, especially for QA?

nlp-question-answering huggingface-transformers gpt-3 deep-learning nlp
1 Answers

Unfortunately GPT-3 and GPT-J both have a 2048 token context limitation, and there's nothing you can do about it.

On my NLP Cloud API, the solution I suggest in general is to fine-tune GPT-J. Fine-tuning GPT-J is like giving ton of context to the model.

user_61918550
Related