Tensorflow doubts about the (input) dimensions of a model when predicting the class of a sequence

user_11579184 0
Viewed 101

I'm new to Tensorflow and I'm trying to implement an irony detection model. My dataset consists of tweets labeled with a 1 or a 0 to indicate whether they're ironic or not.

After the preprocessing, tokenization and padding phases I'm left with fixed-length sequences and an associated vector of labels to split in training and test set and feed as input to the model. The sequences are in the following form:

>>> data
array([[    1,   677,   348, ...,     0,     0,     0],
       [    1,   677,   348, ...,     0,     0,     0],
       [    1,   825,     1, ...,     0,     0,     0],
       ...,
       [  908,  1376,   686, ...,     0,     0,     0],
       [    8,   158, 14579, ...,     0,     0,     0],
       [    1,     1,    35, ...,     0,     0,     0]], dtype=int32)

>>> data.shape
(3977, 50)
>>> data[0].shape
(50,)

The model is the following one:

num_words = len(tok.word_index) + 1 # tok is a Tokenizer which I fit on the data

import tensorflow as tk
from tensorflow import keras
early_stopping = keras.callbacks.EarlyStopping(monitor='val_loss', patience=2)

# The model
model = keras.Sequential()
model.add(keras.layers.Embedding(num_words, 64, input_length=Config.SEQUENCE_LENGTH, mask_zero=True))
model.add(keras.layers.GRU(64, return_sequences=True))
model.add(keras.layers.GRU(64))
model.add(keras.layers.Dense(1, activation='sigmoid'))

After compiling the model with model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) and splitting the dataset with sklearn's utility functions I call the fit method of the model:

model.fit(x_train, y_train, batch_size=10, epochs=10, validation_split=0.1, callbacks=[early_stopping])

After training the model, the evaluate method works as expected, taking as input x_test and y_test but if I call the model.predict_classes(x_test[0]) (or (model.predict(x_test[0]) > 0.5).astype("int32")) instead of a single prediction I get a (50,1)-shaped array of predictions. I tried reshaping x_test[0] in this way model.predict_classes(x_test[0].reshape(1,50)) and I get a single prediction inside an array: array([[1]], dtype=int32)

So now I'm left with the following questions (also due to the fact that I get a 0.6 accuracy when calling evaluate(x_test, y_test):

  • Why if I pass the dataset to the model as an array of arrays (x_train), I cannot just pass an element of the test set to the predict function (e.g. x_test[0]) but I have to reshape it instead?
  • Is it normal or is there some error? Am I setting the input dimensions of the model wrong? Should I reshape the sequences also before feeding them to the model?
keras tensorflow machine-learning python
1 Answers

Why if I pass the dataset to the model as an array of arrays (x_train), I cannot just pass an element of the test set to the predict function (e.g. x_test[0]) but I have to reshape it instead?

It is because model.predict can only accept a set of examples rather than a single example and if you want to supply a single example into it, you have to reshape it into (1,50), so that it is a set of example where the size of the set is 1.

However, instead of feeding in examples one by one into model.predict, it is more efficient to feed in a set of examples into model.predict, i.e. pred_test = model.predict(x_test) and afterwards if you want to know the prediction of the ith example, do prediction_of_the_ith_example = pred_test[i]

Is it normal or is there some error? Am I setting the input dimensions of the model wrong? Should I reshape the sequences also before feeding them to the model?

The model definition and the training part is correct.

Regarding the shape of labels(y_train and y_test), the correct shape to use is (data_size,label_length), i.e. (3181,1) and (796,1) respectively. You are doing single label classification here, so that the label length is 1. But even if you used (3181,) and (796,), it worked properly just because it did automatic broadcasting for you.

user_55239200
Related