Keras retrieve value of node before activation function

So this is for fellow googlers, the working of the keras API has changed significantly since the accepted answer was posted. The working code for extracting a layer's output before activation (for tensorflow backend) is:

model = Your_Keras_Model()
the_tensor_you_need = model.output.op.inputs[0] #<- this is indexable, if there are multiple inputs to this node then you can find it with indexing.

In my case, the final layer was a dense layer with activation softmax, so the tensor output I needed was <tf.Tensor 'predictions/BiasAdd:0' shape=(?, 1000) dtype=float32>.

(TF backend) Solution for Conv layers.

I had the same question, and to rewrite a model's configuration was not an option. The simple hack would be to perform the call function manually. It gives control over the activation.

Copy-paste from the Keras source, with self changed to layer. You can do the same with any other layer.

def conv_no_activation(layer, inputs, activation=False):

    if layer.rank == 1:
        outputs = K.conv1d(
            inputs,
            layer.kernel,
            strides=layer.strides[0],
            padding=layer.padding,
            data_format=layer.data_format,
            dilation_rate=layer.dilation_rate[0])
    if layer.rank == 2:
        outputs = K.conv2d(
            inputs,
            layer.kernel,
            strides=layer.strides,
            padding=layer.padding,
            data_format=layer.data_format,
            dilation_rate=layer.dilation_rate)
    if layer.rank == 3:
        outputs = K.conv3d(
            inputs,
            layer.kernel,
            strides=layer.strides,
            padding=layer.padding,
            data_format=layer.data_format,
            dilation_rate=layer.dilation_rate)

    if layer.use_bias:
        outputs = K.bias_add(
            outputs,
            layer.bias,
            data_format=layer.data_format)

    if activation and layer.activation is not None:
        outputs = layer.activation(outputs)

    return outputs

Now we need to modify the main function a little. First, identify the layer by its name. Then retrieve activations from the previous layer. And at last, compute the output from the target layer.

def get_output_activation_control(model, images, layername, activation=False):
    """Get activations for the input from specified layer"""

    inp = model.input

    layer_id, layer = [(n, l) for n, l in enumerate(model.layers) if l.name == layername][0]
    prev_layer = model.layers[layer_id - 1]
    conv_out = conv_no_activation(layer, prev_layer.output, activation=activation)
    functor = K.function([inp] + [K.learning_phase()], [conv_out]) 

    return functor([images]) 

Here is a tiny test. I'm using VGG16 model.

a_relu = get_output_activation_control(vgg_model, img, 'block4_conv1', activation=True)[0]
a_no_relu = get_output_activation_control(vgg_model, img, 'block4_conv1', activation=False)[0]

print(np.sum(a_no_relu < 0))
> 245293

Set all negatives to zero to compare with the results retrieved after an embedded in VGG16 ReLu operation.

a_no_relu[a_no_relu < 0] = 0
print(np.allclose(a_relu, a_no_relu))
> True

easy way to define new layer with new activation function:

def change_layer_activation(layer):

    if isinstance(layer, keras.layers.Conv2D):

        config = layer.get_config()
        config["activation"] = "linear"
        new = keras.layers.Conv2D.from_config(config)

    elif isinstance(layer, keras.layers.Dense):

        config = layer.get_config()
        config["activation"] = "linear"
        new = keras.layers.Dense.from_config(config)

    weights = [x.numpy() for x in layer.weights]

    return new, weights

I had the same problem but none of the other answers worked for me. Im using a newer version of Keras with Tensorflow so some answers dont work now. Also the structure of the model is given so i can't change it easely. The general idea is to create a copy of the original model that will work exactly like the original one but spliting the activation from the outputs layers. Once this is done we can easely access the outputs values before the activation is applied.

First we will create a copy of the original model but with no activation on the outputs layers. This will be done using Keras clone_model function (See Docs).

from tensorflow.keras.models import clone_model
from tensorflow.keras.layers import Activation

original_model = get_model()

def f(layer):
  config = layer.get_config()
  if not isinstance(layer, Activation) and layer.name in original_model.output_names:
    config.pop('activation', None)
  layer_copy = layer.__class__.from_config(config)
  return layer_copy

copy_model = clone_model(model, clone_function=f)  

This alone will only make a clone with new weights so we must copy the original_model weights to the new one:

copy_model.build(original_model.input_shape)
copy_model.set_weights(original_model.get_weights())

Now we will add the activations layers:

from tensorflow.keras.models import Model

old_outputs = [ original_model.get_layer(name=name) for name in copy_model.output_names ]
new_outputs = [ Activation(old_output.activation)(output) if old_output.activation else output 
                for output, old_output in zip(copy_model.outputs, old_outputs) ]
copy_model = Model(copy_model.inputs, new_outputs)

Finally we could create a new model whose evaluation will be the outputs with no activation applied:

no_activation_outputs = [ copy_model.get_layer(name=name).output for name in original_model.output_names ]
no_activation_model = Model(copy.inputs, no_activation_outputs)

Now we could use copy_model like the original_model and no_activation_model to access pre-activation outputs. Actually you could even modify the code to split a custom set of layers instead of the outputs.