Issues with embeddings for categorical variables in Keras

Viewed 1841

So i have a dataframe with 1000 rows and 6 columns. three columns are categorical variables, and 2 are continuous. The last column is what im trying to predict as have values 1 and 0.

Im trying to use keras to create embeddings for the categorical variables, a dense layer for every continuous variable, concatenate them and fit the model to the data.

The first thing i did was label encoded the categorical variables.

Then I looped over the categorical variables as follows to create a model and appended to a list as follows :-

models_concat = []

for k in df[categorical_columns]:

s = Sequential()

s.add(Embedding(len(df[k].unique()), 5, input_length = 1, embeddings_regularizer = l2(1e-2)))

s.add(Flatten())

models_concat.append(s)

And for the continous variables:-

for i in df[cont_vars]:

s = Sequential()

s.add(Dense(1, input_dim = 1))

models_concat.append(s)

Then I tried to concatenate them as follows and built the network:-

model_emb = Sequential()

model_emb.add(Merge(models_concat, mode='concat'))

model_emb.add(Dropout(0.02))

model_emb.add(Dense(units=100, kernel_initializer= trunc_normal))

model_emb.add(Dropout(0.25))

model_emb.add(BatchNormalization())

model_emb.add(Activation('relu'))

model_emb.add(Dense(units=100, kernel_initializer = trunc_normal))

model_emb.add(Dropout(0.2))

model_emb.add(BatchNormalization())

model_emb.add(Activation('relu'))

model_emb.add(Dense(units=50, kernel_initializer = trunc_normal))

model_emb.add(Dropout(0.1))

model_emb.add(BatchNormalization())

model_emb.add(Activation('relu'))

model_emb.add(Dense(units=1, kernel_initializer = trunc_normal))

model_emb.add(Activation('sigmoid'))

model_emb.compile(optimizer='adagrad',loss='binary_crossentropy',metrics=['accuracy'])

But when I try to fit the model as follows:-

model_emb.fit(train_x,train_y,epochs = 20,validation_split = 0.3,batch_size = 64)

I get the following error :-

ValueError: The model expects 5 input arrays, but only received one array. Found: array with shape (1000, 5).

Any idea as to what's happening and how I could rectify this issue.

P.S -> I'm not sure this method is the right way to do stuff, I was trying to experiment using embeddings in categorical as followed in this paper :-

https://arxiv.org/pdf/1604.06737.pdf

Thanks.

1 Answers

This should be as simple as dividing your training data into a list of 5 numpy.arrays (as your model has 5 different inputs). Try the following code:

train_x = [train_x[:, [i]] for i in range(5)]
Related