I am trying to compute a "summary embedding vector" using the huggingface pre-trained BART model. After I feed in a text to summarise, can I take the last hidden state of the decoder from the pre-trained BART model and normalise that vector to compute a "summary embedding vector"? Or I should take the encoder last hidden state of my input and normalise that instead? From my understanding, the encoder is useful to extract information about the input, and hence taking the last hidden state of encoder would give me a representation of the text I wish to summarise. Does anyone have any suggestions? Thanks a lot!
0 Answers
Related