call method in custom keras layers not covered in pytest

user_4044400 0
Viewed 94

I am trying to unit test a keras neural network that uses custom layers. When I try and run it through pytest and coverage, it states that all of the code inside the call method is not covered.

I don't know if this is due to an idiosyncrasy of how pytest delivers the model to the unit tests or if the call method is really not being used in my neural network, and hence something is wrong.

Here's some example code:

Sample Neural Network

class Linear(keras.layers.Layer):
    def __init__(self, units=32, input_dim=32):
        super(Linear, self).__init__()
        self.w = self.add_weight(
            shape=(input_dim, units), initializer="random_normal", trainable=True
        )
        self.b = self.add_weight(shape=(units,), initializer="zeros", trainable=True)

def call(self, inputs):
    return tf.matmul(inputs, self.w) + self.b

def build_keras_model():
    
    inputs = keras.layers.Input(shape = (32,))
    x = Linear()(inputs)
    outputs = keras.layers.Dense(1)(x)
    
    model = keras.Model(inputs, outputs)
    return model

Sample PyTest File

from model import build_keras_model
import pytest
import numpy as np

def test_keras_model():
    model = build_keras_model()
    
    X = np.random.normal(size = (100, 32))
    y = np.random.normal(size = 100)[:, None]
    
    model.compile(loss = 'mae')
    
    model.fit(X, y, epochs = 1)
    
    assert model.predict(X).shape == (100, 1)

If I test this code in its own directory with coverage run -m pytest and then coverage report -m, then it shows the line inside the call method is not covered.

Is the issue with pytest, keras, or the way I set things up?

keras tensorflow pytest python
1 Answers

I think you have to explicitly call the __call__() method of model if you plan to have 100% test coverage. Interestingly, model.predict does not seem to do that (at least, not at the abstraction level that you need. It does also use the same method internally). Try this:

%%file test_keras.py
import pytest
import numpy as np
import tensorflow as tf

class Linear(tf.keras.layers.Layer):
    def __init__(self, units=32, input_dim=32):
        super(Linear, self).__init__()
        self.w = self.add_weight(
            shape=(input_dim, units), initializer="random_normal", trainable=True
        )
        self.b = self.add_weight(shape=(units,), initializer="zeros", trainable=True)

    def call(self, inputs):
        return tf.matmul(inputs, self.w) + self.b

def build_keras_model():
    
    inputs = tf.keras.layers.Input(shape = (32,))
    x = Linear()(inputs)
    outputs = tf.keras.layers.Dense(1)(x)
    
    model = tf.keras.Model(inputs, outputs)
    return model

def test_keras_model():
    model = build_keras_model()
    
    X = np.random.normal(size = (100, 32))
    y = np.random.normal(size = 100)[:, None]
    
    model.compile(loss = 'mae')
    
    model.fit(X, y, epochs = 1)
    
    assert model(X).shape == (100, 1)

!coverage run -m pytest

============================= test session starts ==============================
platform linux -- Python 3.7.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /content
plugins: cov-3.0.0, typeguard-2.7.1
collected 1 item                                                               

test_keras.py .                                                          [100%]

============================== 1 passed in 6.46s ===============================

!coverage report -m /content/test_keras.py

Name            Stmts   Miss  Cover   Missing
---------------------------------------------
test_keras.py      23      0   100%
---------------------------------------------
TOTAL              23      0   100%

Check this post on the differences between the call method and predict method and this.

user_96578610
Related