Model returning error - ValueError: logits and labels must have the same shape ((None, 18) vs (None, 1))

user_640558 0
Viewed 8688

I'm playing around with a keras based multi label classifer. I created a function that loads training and test data and then I process/split X/Y within the function itself. I'm getting a error when running my model but not quite sure the meaning:

Here's my code:

def KerasClassifer(df_train, df_test):
  X_train = df_train[columnType].copy()
  y_train = df_train[variableToPredict].copy()
  labels = y_train.unique()
  print(X_train.shape[1])
  #using keras to do classification
  from tensorflow import keras
  from tensorflow.keras.models import Sequential
  from tensorflow.keras.layers import Dense, Dropout, Activation
  from tensorflow.keras.optimizers import SGD

  model = Sequential()
  model.add(Dense(5000, activation='relu', input_dim=X_train.shape[1]))
  model.add(Dropout(0.1))
  model.add(Dense(600, activation='relu'))
  model.add(Dropout(0.1))
  model.add(Dense(len(labels), activation='sigmoid'))

  sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
  model.compile(loss='binary_crossentropy',
                optimizer=sgd)

  model.fit(X_train, y_train, epochs=5, batch_size=2000)

  preds = model.predict(X_test)
  preds[preds>=0.5] = 1
  preds[preds<0.5] = 0

  score = model.evaluate(X_test, y_test, batch_size=2000)
  score

Here are attributes of my data(if it helps):

x train shape  (392436, 109)
y train shape  (392436,)
len of y labels 18

How can I fix the code to avoid this error?

keras tensorflow python
2 Answers

If you have 18 categories the shape of y_train should be (392436, 18). You can use tf.one_hot for that:

import tensorflow as tf

y_train = tf.one_hot(y_train, depth=len(labels))

And if you're taking your values from one column, I suspect this is not "multi-label", but multi-class. Can a sample really belong to multiple categories? If not, you will need to change a few other things too. For instance, you would need softmax activation:

model.add(Dense(len(labels), activation='softmax'))

And also categorical crossentropy loss:

model.compile(loss='categorical_crossentropy', optimizer=sgd)
user_109083750

your labels are 1D... you need to one-hot encode them if your aim is to operate a multi label task. You can simple use

tf.keras.utils.to_categorical(y, num_classes)

in this way you transform your target from (n_sample) to (n_sample, num_classes)

if it's a simple multi class problem instead you have these 2 possibilities:

1 possibility) if you have 1D integer encoded target, you can use sparse_categorical_crossentropy as loss function with softmax activation to generate probabilities

n_class = 3
n_features = 100
n_sample = 1000

X = np.random.randint(0,10, (n_sample,n_features))
y = np.random.randint(0,n_class, n_sample)

inp = Input((n_features,))
x = Dense(128, activation='relu')(inp)
out = Dense(n_class, activation='softmax')(x)

model = Model(inp, out)
model.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
history = model.fit(X, y, epochs=3)

pred = np.argmax(model.predict(X), 1)

2 possibility) if you have one-hot encoded your target in order to have 2D shape (n_samples, num_classes), you can use categorical_crossentropy with softmax activation to generate probabilities

n_class = 3
n_features = 100
n_sample = 1000

X = np.random.randint(0,10, (n_sample,n_features))
y = tf.keras.utils.to_categorical(np.random.randint(0,n_class, n_sample))

inp = Input((n_features,))
x = Dense(128, activation='relu')(inp)
out = Dense(n_class, activation='softmax')(x)

model = Model(inp, out)
model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])
history = model.fit(X, y, epochs=3)

pred = np.argmax(model.predict(X), 1)
user_103750490
Related