Doc2Vec : Paragraph matrix (D) in the structure of the PV-DBOW model

user_16373428 0
Viewed 118

I am confused about the meaning of paragraph matrix (D) in the structure of the PV-DBOW model (Doc2Vec).

Is the paragraph matrix the result of one-hot encoding of n input paragraph IDs?

Or is the paragraph matrix a randomly initialized weight to generate the shape(nxp), where n is the number of paragraph ID inputs and p is the vector dimension?

PV-DBOW

doc2vec word-embedding embedding
1 Answers

I've never found the original paper's diagrams very clear or helpful. (And, no one has been able to reproduce their claimed results in the 'concatenation' mode.)

But I can say, from familiarity with code implementations:

  • At the outset, every paragraph-ID gets a randomly-initialized vector. Thus, there is a matrix in the model with all of these vectors, of shape number_of_paragraph_ids x number_of_dimensions
  • For backprop-training in PV-DBOW, each individual paragraph-vector (one row from the above matrix) is adjusted to better predict the matching-paragraph's individual constituent words.

While figuratively, that's sort of a 1-hot selection of a single paragraph choice, in the code it's just a lookup of the single correct row using a paragraph-ID key.

user_1302880
Related