StringLookup equivalent for tensorflow v2.1.0

user_8349044 0
Viewed 771

I am trying to build one recommendation model similar to this example. But this example uses Tensorflow v2.4.0 and for my work, I need to use v2.1.0. It seems that the StringLookUp layer does not exist in v2.1.0. Is there any equivalent way to achieve the exact same thing in 2.1.0? I need to use this in such a model:

user_model = tf.keras.Sequential([
  tf.keras.layers.experimental.preprocessing.StringLookup(
      vocabulary=unique_user_ids, mask_token=None),
  tf.keras.layers.Embedding(len(unique_user_ids) + 1, embedding_dimension)
])
tensorflow2.0 tensorflow
1 Answers

You can use tf.strings.to_hash_bucket_strong to hash your strings to indices, as long as you don't care about the mapping order.

Example:

import tensorflow as tf

print(tf.__version__)
# 2.1.0

NUM_BUCKETS = 6
EMB_DIM = 128

# five unique user ids
user_ids = tf.constant([u'463', u'112', u'666', u'932', u'878', u'[UNK]'])

# hash to numbers 0-5; 5 numbers for the known unique ids and one
# to account for unknown or empty strings
idxs = tf.strings.to_hash_bucket_strong(
    input=user_ids,
    num_buckets=NUM_BUCKETS,
    key=[1, 2])

print(idxs)
# <tf.Tensor: shape=(6,), dtype=int64, numpy=array([2, 3, 4, 0, 5, 1])>

# And now you can apply your embeddings to the indices you've generated
emb = tf.keras.layers.Embedding(
    input_dim=NUM_BUCKETS,
    output_dim=EMB_DIM)

assert emb(idxs).shape == (6, 128)
user_32227970
Related