Dense layer binary classification cannot be set to 2

user_3276223 0
Viewed 352

I'm fairly new to keras and tensorflow. I'm trying to figure out why running my code gives me an error when using dense layer = 2 and not dense = 1. This is how I assigned the classes based on a dir structure:

res_scans = np.array([process_scan(path) for path in res_scan_paths])
non_res_scans = np.array([process_scan(path) for path in non_res_scan_paths])

# For the CT scans having presence of viral pneumonia
# assign 1, for the normal ones assign 0.
res_labels = np.array([1 for _ in range(len(res_scans))])
non_res_labels = np.array([0 for _ in range(len(non_res_scans))])

#Split data 
spl = 0.7
resF = round(spl * len(res_scan_paths) )
non_resF = round(spl * len(non_res_scan_paths))


# Split data in the ratio 70-30 for training and validation.
x_train = np.concatenate((res_scans[:resF], non_res_scans[:non_resF]), axis=0)
y_train = np.concatenate((res_labels[:resF], non_res_labels[:non_resF]), axis=0)
x_test = np.concatenate((res_scans[resF:], non_res_scans[non_resF:]), axis=0)
y_test = np.concatenate((res_labels[resF:], non_res_labels[non_resF:]), axis=0)

This is the CNN model used a VGG16 with batch normalization between conv layers

#VGG16 Model
lr = 0.001
activ='relu'

Model = Sequential()
Model.add(Conv3D(64, (3,3,3), strides=(1, 1, 1), input_shape = (91, 109, 91, 30), activation =activ, padding="same"))
Model.add(Conv3D(64, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(MaxPool3D((2,2,2), strides = (2,2,2)))
  
Model.add(Conv3D(128, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(Conv3D(128, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(MaxPool3D((2,2,2), strides = (1,1,1)))

Model.add(Conv3D(256, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(Conv3D(256, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization())
Model.add(Conv3D(256, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(MaxPool3D((2,2,2), strides = (2,2,2)))

Model.add(Conv3D(512, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(Conv3D(512, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization())
Model.add(Conv3D(512, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(MaxPool3D((2,2,2), strides = (2,2,2)))

Model.add(Conv3D(512, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(Conv3D(512, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization())
Model.add(Conv3D(512, (3,3,3), strides=(1, 1, 1), activation =activ, padding="same"))
Model.add(BatchNormalization()) 
Model.add(MaxPool3D((2,2,2), strides = (1,1,1)))

Model.add(Flatten())
Model.add(Dense(units=4096,activation="relu"))
Model.add(Dense(units=4096,activation="relu"))
Model.add(Dense(units=2, activation="softmax"))

Model.summary()

And then compiling and runnning it below

Model.compile(loss='mse' , optimizer=tf.keras.optimizers.Adam(lr=lr), metrics=["acc"])
history = Model.fit(x_train, y_train, epochs=30, batch_size=12, verbose=1,validation_data=(x_test, y_test))

I get this error which perplexes me:

ValueError: A target array with shape (18, 1) was passed for output of shape (None, 2) while using as loss `mean_squared_error`. This loss expects targets to have the same shape as the output.

I can run this fine on another computer using only CPU. However on the computer with 2 bridged 2080ti it gives me that error. When i changed it to 1 it gives me dead kernel after first epoch. I know from a previous post dense=2 is the correct way about it. Please help.

kevin

keras tensorflow deep-learning machine-learning python
2 Answers

From the following statement of yours, it seems like you're trying to build a binary classification model.

# For the CT scans had the presence of viral pneumonia
# assign 1, for the normal ones assign 0.

That's why the correct setting would be as follows:

...
# (1)
# last layer of your model should have 1 output unit and
# with sigmoid activation
Model.add(Dense(units=1, activation="sigmoid"))

# (2)
# and your loss function should be as follows
Model.compile(loss='binary_crossentropy' , 
              optimizer=tf.keras.optimizers.Adam(lr=lr), 
              metrics=["acc"])

Now, you can train your model and the problem should be resolved.

More info for you

You pick mse as a loss function for units=2, activation="softmax", which is wrong. Generally, for mse, the activation should be None that means linear.

...
Model.add(Dense(units=1, activation=None))

Model.compile(loss='mse' , 
              optimizer=tf.keras.optimizers.Adam(lr=lr), 
              metrics=["acc"])

Also, choosing activation to softmax, your loss function should be categorical_crossentropy. But note that in your case the target is not 2D. For your case it's a binary target [0, 1, 0] and not in this shape: [[0, 1], [1,0], [0,1]]. If you convert your target from [0, 1, 0] to [[0, 1], [1,0], [0,1]], then you can need to set up as follows. But note that, that makes no sense because your target is already one-hot encoded.

...
Model.add(Dense(units=2, activation='softmax'))

Model.compile(loss='categorical_crossentropy' , 
              optimizer=tf.keras.optimizers.Adam(lr=lr), 
              metrics=["acc"])
user_92157800

The output of your neural network should be suitable for your y-label size in the dataset. It seems your y-label is an 18 element vector, so you should have 18 neurons in the output layer. OR, you should change the dataset y-label if you think it shouldn't be composed of 18 elements. In MAE:

loss = mean(abs(y_true - y_pred), axis=-1)

So, the output of the neural network y_pred should have same size as your dataset y-label (that is, y_true)

user_143665060
Related