Freeze model and train it

I am using Keras Tensorflow in Colab. I fit a model and save it. Then I load it and check the performance and of course it should be the same. Then I freeze it and I fit it again. I would expect that afterwards the model has the same performance. Of course during "training" due to batch size differences there can be differences in the accuracy. But afterwards when checking it with model.evaluate I would expect no differences, as the weights cannot be changed, as the model was frozen. However, it turns out this is not the case.

My code:

import csv
import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

(train_x, train_labels), (test_x, test_labels) = tf.keras.datasets.imdb.load_data(num_words=10000)

x_train_padded = pad_sequences(train_x, maxlen=500)
x_test_padded = pad_sequences(test_x, maxlen=500)

model = tf.keras.Sequential([
    tf.keras.layers.Embedding(10000, 128, input_length=500),
    tf.keras.layers.Conv1D(128, 5, activation='relu'),
    tf.keras.layers.GlobalAveragePooling1D(),
    tf.keras.layers.Dense(64, activation='relu'),
    tf.keras.layers.Dense(1)
])

model.compile(loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),optimizer='adam', metrics=[tf.metrics.BinaryAccuracy(threshold=0.0, name='accuracy')])

history = model.fit(x=x_train_padded,
                      y=train_labels,
                      validation_data=(x_test_padded , test_labels),
                      epochs=4, batch_size=128)

gives the output:

I save the model:

model.save('test.h5')

and load it back:

modelloaded=tf.keras.models.load_model('test.h5')

and check the performance:

modelloaded.evaluate(x_test_padded , test_labels)

of course still the same:

Now I set the model to non-trainable:

modelloaded.trainable=False

and indeed:

modelloaded.summary()

shows that all parameters are non-trainable:

Now I fit it again, using just one epoch:

history = modelloaded.fit(x=x_train_padded,
                      y=train_labels,
                      validation_data=(x_test_padded , test_labels),
                      epochs=1, batch_size=128)

I understand that although the weights are non-trainable, the accuracy changes as this depends on the batch size.

However, when I check the model afterwards with:

modelloaded.evaluate(x_test_padded , test_labels)

I can see that the model was changed? The loss and the accuracy is different. I do not understand why, I would have expected the same numbers. As the model cannot be trained. It doesn't matter if I call it with different batch sizes:

modelloaded.evaluate(x_test_padded , test_labels, batch_size=16)

The numbers are always the same, however different to those before the model fitting.

Edit:

I tried the following:

modelloaded=tf.keras.models.load_model('test.h5')
modelloaded.trainable=False

for layer in modelloaded.layers:
  layer.trainable=False

history = modelloaded.fit(x=x_train_padded,
                          y=train_labels,
                          validation_data=(x_test_padded , test_labels),
                          epochs=1, batch_size=128)

modelloaded.evaluate(x_test_padded, test_labels)

However, still the weights are adjusted (I checked this with comparing print(modelloaded.trainable_variables) before and afterwards) and the modelloaded.evaluate output gives slightly different results, where I would expect no changes. As the model weights should not have changed. But they did, as I can see when checking print(modelloaded.trainable_variables).