Correctly preprocess csv data for 1D CNN

user_2190822 0
Viewed 536

I've a problem preparing my dataset to feed a 1D CNN.

My CSV has 3025 cols representing a single byte + last col as string label.

Maybe is not a problem of preprocessing but on my network model.

This is my model:

def cnn_1d(num_classes):
    model = models.Sequential()
    model.add(Conv1D(16, kernel_size=3, strides=1, activation="relu", input_shape=(3025, 1)))
    model.add(Conv1D(16, kernel_size=3, strides=1, activation="relu"))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Dropout(0.2))
    model.add(Conv1D(32, kernel_size=3, strides=1, activation="relu"))
    model.add(Conv1D(32, kernel_size=3, strides=1, activation="relu"))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Dropout(0.2))
    model.add(Dense(500, activation="relu"))
    model.add(Dense(300, activation="relu"))
    model.add(Dense(num_classes, activation="softmax"))
    model.compile(
        optimizer=keras.optimizers.Adam(1e-3),
        loss="categorical_crossentropy",
        metrics=["accuracy"],
    )
    model.summary()
    return model

This is what I tried to preprocess data:

num_classes = 4
df = pd.read_csv("test.csv", header=0)

df["label"] = pd.Categorical(df["label"])
df["label"] = df.label.cat.codes

Y = df.pop("label")
X = df.copy()

x_train, x_test, y_train, y_test = train_test_split(np.asarray(X), np.asarray(Y), test_size=0.33, shuffle=True)

x_train = np.reshape(x_train, (x_train.shape[0], x_train.shape[1], 1))
x_test = np.reshape(x_test, (x_test.shape[0], x_test.shape[1], 1))

model = cnn_1d(num_classes)
model.fit(x_train, y_train, epochs=100, batch_size=64, validation_data=(x_test, y_test))

I think I get an error on last Dense layer due to uncorrectly preprocessed labels. This i

 ValueError: Shapes (None, 1) and (None, 753, 4) are incompatible

What I'm missing? What I know is that last Dense layer should have num classes as unit count (4 in my example).

conv-neural-network keras tensorflow convolution python
2 Answers

This is the model summary of the code you present above:

Model: "sequential_1"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d (Conv1D)              (None, 3023, 16)          64        
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 3021, 16)          784       
_________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 1510, 16)          0         
_________________________________________________________________
dropout (Dropout)            (None, 1510, 16)          0         
_________________________________________________________________
conv1d_2 (Conv1D)            (None, 1508, 32)          1568      
_________________________________________________________________
conv1d_3 (Conv1D)            (None, 1506, 32)          3104      
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 753, 32)           0         
_________________________________________________________________
dropout_1 (Dropout)          (None, 753, 32)           0         
_________________________________________________________________
dense (Dense)                (None, 753, 500)          16500     
_________________________________________________________________
dense_1 (Dense)              (None, 753, 300)          150300    
_________________________________________________________________
dense_2 (Dense)              (None, 753, 4)            1204      
=================================================================
Total params: 173,524
Trainable params: 173,524
Non-trainable params: 0

The output layer has a dimensions of (batch, sequence length, 4 classes). You probably meant to flatten the matrix after the second max_pooling layer.

If I do that I get a model that has less parameters and will output one of 4 classes...

def cnn_1d(num_classes):
    model = models.Sequential()
    model.add(Conv1D(16, kernel_size=3, strides=1, activation="relu", input_shape=(3025, 1)))
    model.add(Conv1D(16, kernel_size=3, strides=1, activation="relu"))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Dropout(0.2))
    model.add(Conv1D(32, kernel_size=3, strides=1, activation="relu"))
    model.add(Conv1D(32, kernel_size=3, strides=1, activation="relu"))
    model.add(MaxPooling1D(pool_size=2))
    model.add(Flatten())
    model.add(Dropout(0.2))
    model.add(Dense(500, activation="relu"))
    model.add(Dense(300, activation="relu"))
    model.add(Dense(num_classes, activation="softmax"))
    model.compile(
        optimizer=keras.optimizers.Adam(1e-3),
        loss="categorical_crossentropy",
        metrics=["accuracy"],
    )
    model.summary()
    return model
  
cnn_1d(4)

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_4 (Conv1D)            (None, 3023, 16)          64        
_________________________________________________________________
conv1d_5 (Conv1D)            (None, 3021, 16)          784       
_________________________________________________________________
max_pooling1d_2 (MaxPooling1 (None, 1510, 16)          0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 1510, 16)          0         
_________________________________________________________________
conv1d_6 (Conv1D)            (None, 1508, 32)          1568      
_________________________________________________________________
conv1d_7 (Conv1D)            (None, 1506, 32)          3104      
_________________________________________________________________
max_pooling1d_3 (MaxPooling1 (None, 753, 32)           0         
_________________________________________________________________
flatten (Flatten)            (None, 24096)             0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 24096)             0         
_________________________________________________________________
dense_3 (Dense)              (None, 500)               12048500  
_________________________________________________________________
dense_4 (Dense)              (None, 300)               150300    
_________________________________________________________________
dense_5 (Dense)              (None, 4)                 1204      
=================================================================
Total params: 12,205,524
Trainable params: 12,205,524
Non-trainable params: 0

As a bonus this model has significantly fewer trainable parameters.

user_52989370

I figured out what I was missing:

  1. I missed to use to_categorical on y_* variable. I thought that it was alredy categorical with df["label"] = pd.Categorical(df["label"]). So before model I added: 
    y_train = to_categorical(y_train, 4)
y_test = to_categorical(y_test, 4)
  2. I forgot to Flatten output after last MaxPool1D layer.

Now it works correctly.

user_21908220
Related