Keras: Changing the structure of a trained model

user_76701 0
Viewed 1377

I'm a newbie at machine learning. I'm learning it using Keras by giving myself exercises.

Here is the video from my latest exercise. I've trained a convolutional neural network to identify three different objects. The phone sends the image to the web server on my desktop, which runs Keras.

It took a while until I got a model that worked. I started by training on video frames of these 3 objects against a sterile background. I don't know if there's an accepted name for this technique, but I used a sort of "training wheels" approach. First I trained the model on a really easy background. Then took more videos on backgrounds that are busier. I did 6 rounds, each round against a different background. At each round, I loaded the weights from the previous rounds, and trained the network on the combined data set from all the previous rounds. The idea was to get the neural net started on something that's easy, and then gradually expand its knowledge rather than try to give it a difficult task right away.

That seems to have worked, I have good accuracy now when identifying these objects on different backgrounds. I've saved the model weights to a file.

My question is: Now that I have good trained network, is it possible to change its structure? For example right now I'm cropping and resizing the images to 64x64. If I wanted to up it to 128x128, will I be able to use the weights I've trained so far? What if I wanted to add color (so 3 channels instead of 1), is that possible without starting the training from scratch?

keras tensorflow machine-learning neural-network python
1 Answers

Understanding a little bit about neural networks. The first layers of your neural network architecture learn simple features.
For eg
Consider images the first layers will learn to detect horizontal or vertical edges and later on it will be able to detect more complex features. Take a look at this and this.

Now, you have a trained neural network with good accuracy and you want to use these trained weights but at the same time change the architecture.

Keeping the top part of model constant and changing the architecture of bottom layers:

This approach will work if you keep the top layers as they are and change the architecture of lower layers. ie. add/remove dense layers, add/remove convolution layers.

This is because the features learned in the previous layer are responsible for the features of the next layer. Take look at this. If you change the number of neurons in the first layer the feature representation changes in the following layers.

So, if you add another convolutional layer to your existing layer it will take the feature generated from the trained model and use these features to learn even more complex features.

Now if you want to keep the top part of your model constant you have to use the same input format your model is trained on.

Changing the dept:
When you change the input the number of parameters as well as the type of layers that can handle that specific input changes.

model = tf.keras.models.Sequential([
  tf.keras.layers.Conv2D(3, 3,input_shape=(64,64,1)),
  tf.keras.layers.Dense(512, activation=tf.nn.relu),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.summary()

Output:

Model: "sequential_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_5 (Conv2D)            (None, 62, 62, 3)         30        
_________________________________________________________________
dense_10 (Dense)             (None, 62, 62, 512)       2048      
_________________________________________________________________
dense_11 (Dense)             (None, 62, 62, 10)        5130      
=================================================================
Total params: 7,208
Trainable params: 7,208
Non-trainable params: 0

Code:

model = tf.keras.models.Sequential([
  tf.keras.layers.Conv2D(3, 3,input_shape=(64,64,3)),
  tf.keras.layers.Dense(512, activation=tf.nn.relu),
  tf.keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.summary()

Output:

Model: "sequential_6"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_6 (Conv2D)            (None, 62, 62, 3)         84        
_________________________________________________________________
dense_12 (Dense)             (None, 62, 62, 512)       2048      
_________________________________________________________________
dense_13 (Dense)             (None, 62, 62, 10)        5130      
=================================================================
Total params: 7,262
Trainable params: 7,262
Non-trainable params: 0

As you can see the parameters are different for both the models. Therefore, you won't be able to use the weights from one model into your second model since the dimensions of the weight matrix won't match.

Changing input size:

If you change the input size without changing the number of channels ie. (64,64,1) --> (128, 128, 1) your parameters will remain the same. So you can use the previously trained weights with the new model.
However, know that the weights learned are dependent on the size of the input. So what worked well for (64,64,1) won't necessarily work for (128,128,1) but can act as good initialization weights.

Adding untrained layer before your trained architecture:

This won't work because as we have seen earlier the features generated from the previous layers are responsible for the features of the next layer. If the features of previous layers change the features of the next layers should change accordingly.

Key Points:

  1. If you want to utilize a pre-trained model keep the model as a top part of your new architecture and use the same input structure that used for training the pre-trained model.

  2. If you want to change the number of channels you are better off training your model from scratch.

  3. If you want to change your input size you can use the pre-trained model weights as a good starting point.

user_95579700
Related