Generate equidistant multidimensional vectors as Embedding matrix

user_10520866 0
Viewed 193

I need to generate an embedding matrix to use instead of the layer. I know a priori the similarity between the 10 features (all equidistant from each other) and I can't generate the matrices through training because I don't have enough data.

To do this I have to generate 10 vectors of arbitrary size (ie 10) but which all have the same size and which are all equidistant from each other, with values of the single dimensions being numbers between -1 and 1, all this in python.

Anyone know how this can be done?

keras tensorflow embedding vector python
1 Answers

I believe you have some words as features and you want to represent them as embedding vectors.

There are several ways to create word embeddings, I will mention a few of these from the simplest to the complex yet very powerful methods.

1. Count Vector.

It is a method of creating vectors out of your unique tokens. For example, if the vocabulary contains three words, say ["and", "basketball", "more"] , then thetext "more and more" will be mapped to the vector [1, 0, 2] : the word "and" appears once, the word "basketball" does not appear at all, and the word "more" appears twice. This text representation is called a bag of words, since it completely loses the order of the words.

2. TF-IDF( Term Frequency Inverse document frequency)

The problem with the count vector is that it ignores the important word since it is having less appearance compared to common words. In the above example the term "basketball" is ignored and "more" is given importance. To overcome this TF-IDF approach is best suited, For example, let’s imagine that the words "and" , "basketball" , and "more" appear respectively in 200, 10, and 100 text instances in the training set: in this case, the final vector will be [1/log(200), 0/log(10), 2/log(100)] , which is approximately equal to [0.19, 0., 0.43] .

3. Pre-trained word vectors.

These are the embedding vectors trained on millions of text data available on Wikipedia or other general sources, it will be having all the common terms available in English. There are many open-sourced pre-trained word vectors are available some of the, are.

  1. GoogleNews vector.
  2. GloVe
  3. fastText by Facebook.

You can choose the vector dimension based on the model's availability, for example you can choose 50,100,200,300 dimensions vector for each word.

from gensim.models import Word2Vec

#loading the downloaded model
model = Word2Vec.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True, norm_only=True)

#the model is loaded. It can be used to perform all of the tasks mentioned above.

# getting word vectors of a word
dog = model['dog']

For more details and other methods to create word embeddings, you can refer to this beautiful article written by NCC.

Hope this answers your question, Happy Learning!

user_142906970
Related