How to compare two keras conv2D layer with kernel size of 7x7 and 3x3

Setting: I am currently working with the Samsung Exynos v9 NPU. The goal is to run inference with ML models on the NPU. The steps are the following:

1. First, I build and train a keras model.
2. Then, I convert it to either tflite or onnx.
3. Samsung provides a Docker container to convert the tflite or onnx
   model into a compatible format for the NPU

Problem: The NPU has some constraints to run ML models. For instance, to compile a tflite model to an NPU compatible format, the maximum kernel size allowed for Conv2D layers is 3x3. I want to compile the Deeplabv3+ which contains as third layer a Conv2D layer with kernel size 7x7.

Goal: Therefore my goal is to compare the performance/impact of two keras conv2D layers with kernel size of 7x7 and 3x3. If there is no significant performance loss, I can just substitute that layer, and compile the model.

What I have tried: I have created an own dataset with circles (1000 samples) and rectangles (1000 samples). My assumption was that a bigger kernel size might be better to identify shapes like the corner of the rectangles. Sample images look like the following:

I created two simple CNN where the first Conv2D has one time kernel size 7x7 and the other 3x3. Then I trained them on the shapes dataset, and evaluated the model on test images. The model looks like the following:

no_classes = 2

model = Sequential()
model.add(Conv2D(filters=64,
                    kernel_size=(7,7),  # and (3,3)
                    strides=(1,1),
                    padding='valid',
                    activation='relu',
                    input_shape=(512,512,3))
model.add(MaxPooling2D(pool_size=(2,2), strides=(1,1)))
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2), strides=(1,1)))
model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))
model.add(Flatten())
model.add(Dropout(0.5))
model.add(Dense(no_classes, activation='softmax'))

# Compile the model
model.compile(loss=sparse_categorical_crossentropy,
            optimizer=Adam(),
            metrics=['accuracy'])

The evaluation provides the following output:

(7x7) kernel size:

50/50 [==============================] - 2000s 40s/step - loss: 15686.3135 - accuracy: 0.4837
13/13 [==============================] - 100s 8s/step - loss: 1112.4458 - accuracy: 0.4900
Test loss: 1112.44580078125 / Test accuracy: 0.49000000953674316

(3x3) kernel size:

50/50 [==============================] - 1858s 37s/step - loss: 4556.7983 - accuracy: 0.4969
13/13 [==============================] - 95s 7s/step - loss: 0.7123 - accuracy: 0.5475
Test loss: 0.7123353481292725 / Test accuracy: 0.5475000143051147

The results show that the evaluation results with 3x3 are almost 6 % better. Is that a valid approach? Or are there better approaches?

Questions that came up while researching on that topic:

• Can you actually compare the two layers with two different kernel sizes?
• Can you compare them isolated?
• Do they have different impact if they are concatenated with other layers?