Difference between StringLookup layer and embedding layer in tensorflow layers

user_972648 0
Viewed 34

Documentation of StringLookUp says that it encodes categorical features to numerical features, while as per my understanding Embedding layers also do the same . I think only difference is that StringLookUp does a sparse vector transformation in which OOV values are marked at 0, while Embedding layer does a dense vector representation.

Please help me understand the difference between these layers

P.S Improving my question statement.

From what I have understood ,One-hot encodings causes high dimensionality issues for the categorical feature having many different values. However, StringLookUp layer with output_mode =int, will also result in dense list for m words (only one index for one word mx1 matrix), resolving the drawbacks of one_hot encoding.

Still what benefit would I get , using embedding layer over stringlookup layer in a DNN model for supervised problem?

keras tensorflow python
1 Answers

Let's assume you have a StringLookup layer with default settings. It will create an internal dictionary mapping each word to a unique integer value:

StringLookup

the  --> 0
fat  --> 1
cat  --> 2
in   --> 3
your --> 4
hat  --> 5

You could, of course, use the other output modes: one_hot, multi_hot, count, or tf_idf, which will create other representations of your tokens / words. For example, if you were to use the one_hot output mode, you could directly train your model on these representations without an Embedding layer.

What the Embedding layer does, as you mention in your question, is map unique integer values to dense vector representations. However, it is important to note that this layer cannot work directly with strings. That is why you usually map your tokens / words to integer values beforehand. A nice feature of the Embedding layer is that you can also make the vector representations trainable.

An advantage of Embedding layers is that you can use pretrained word vectors that know how to treat words in context, unlike one-hot-encoded embeddings.

user_96578610
Related