Best practice to include categorical features along with sequences in LSTM for sequence prediction?

user_11305480 0
Viewed 412

I have a dataset with sequences and categorical attributes associated with each sequence.

What is the best practice to include these categorical attributes in order to capture sequence variations across different attributes?

An example df

 Sequence      |   Country   |   User  |
[A,B,D,E,F]          USA          U1
[B,C,D,E]            DE           U123
[A,B,F,E,G,H]        USA          U2456
  ...                ...          ....

I want to build an LSTM model which will be able to predict the next event in the sequence.

I know how to build the LSTM model if I am only giving it the sequences as input.

But how would I be able to add the attributes Country and User as well?

If I apply one hot encoding on the sequence data, should I concatenate the one hot encoded feature vectors to it?

Or are there any other best practices here?

lstm keras python
1 Answers

you can use functional model API and concatenate the categorical feature

for example, you have the country as cateogrical feature, you can concatenate it with lstm features as Concatenate()([lstm, country])

from keras.models import Model
from tensorflow.keraslayers import Input,Concatenate,Dense,Activation

sequence= Input(shape=(seq_length,), name='sequence')
lstm = LSTM(300, dropout=0.3, recurrent_dropout=0.3)(sequence)
country   = Input(shape=(1,))
conc = Concatenate()([lstm, country])
dense_layer = Dense(1)(conc)
x_out= Activation('sigmoid')(dense_layer )
model = Model([sequence, country], x_out)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics['accuracy'])

To fit the model, assuming you have reshaped and have prepared sequence column in sequence and other feature(country) in country

 model.fit([sequence,country], y_train)
user_36808310
Related