Tensorflow / Keras sigmoid on single output of dense layer

user_8585195 0
Viewed 838

I have a simple CNN model in tensorflow that takes in an image and predicts a 6 element label vector. The final layer of my mode is therefore Dense(6). Label[0] is supposed to be binary, while Label[1:6] is continuous valued. I therefore want to apply a sigmoid activation to the output layer on only the first node of the output, while leaving the other 5 outputs as is. How do I do this with tensorflow.keras? For simplicity, my model building code currently looks something like:

model = tf.keras.models.Sequential()
model.add(Reshape((image_size, image_size, 1), input_shape = (image_size, image_size))
model.add(Conv2D(8, **parameters))
model.add(BatchNormalization())
model.add(Activation('relu')
Model.add(MaxPool2D())
model.add(Flatten())
model.add(Dense(6))

How do I add to this to use sigmoid activation on the first index of the last layer?

keras tensorflow python
2 Answers

You can define a simple custom Lambda layer and do exactly what you want. Here is an example initially without an activation function. Pay attention to the output:

import tensorflow as tf
tf.random.set_seed(2)

def custom_layer(tensor):
    activated_node = tf.nn.sigmoid(tensor[:, :1])
    return tf.concat([activated_node, tensor[:, 1:]], axis=1)

model = tf.keras.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(6))

model(tf.random.uniform((2, 5)))

<tf.Tensor: shape=(2, 6), dtype=float32, numpy=
array([[-1.1554979 ,  0.29463094,  0.57452184,  0.40530735, -0.15730543,
         0.16329125],
       [-1.1518296 ,  1.2684885 ,  0.50156784,  1.2273686 ,  0.13656075,
        -0.7025717 ]], dtype=float32)>

And now with the custom Lambda layer, which applies an activation function to the first node in your tensor:

import tensorflow as tf
tf.random.set_seed(2)

def custom_layer(tensor):
    activated_node = tf.nn.sigmoid(tensor[:, :1])
    return tf.concat([activated_node, tensor[:, 1:]], axis=1)

model = tf.keras.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(6))
model.add(tf.keras.layers.Lambda(custom_layer, name="activation_layer"))

model(tf.random.uniform((2, 5)))

<tf.Tensor: shape=(2, 6), dtype=float32, numpy=
array([[ 0.23948632,  0.29463094,  0.57452184,  0.40530735, -0.15730543,
         0.16329125],
       [ 0.24015504,  1.2684885 ,  0.50156784,  1.2273686 ,  0.13656075,
        -0.7025717 ]], dtype=float32)>

You can clearly see how the first element of each sample (I am using batch_size=2) is squeezed between 0 and 1.

user_96578610

Say that we get the output of your model as pred, then pred would be a tensor of shape(1, 6), so in order to achieve your objective you can do something like this:

sigmoid_input = pred.numpy()[0][0]
sigmoid_output = tf.keras.activations.sigmoid(sigmoid_input)

So first you need to convert the Tensor to a Numpy ndarray and then access just the first element of your Tensor. After that we pass the new variable sigmoid_input holding that value to a sigmoid as planned.

user_23443630
Related