How to change number of channels to fine tune VGG16 net in Keras

user_2426955 0
Viewed 8667

I would like to fine tune the VGG16 model using my own grayscale images. I know I can fine tune/add my own top layers by doing something like:

base_model = keras.applications.vgg16.VGG16(include_top=False, weights='imagenet', input_tensor=None, input_shape=(im_height,im_width,channels))

but only when channels = 3 according to the documentation. I have thought of simply adding two redundant channels to my image, but this seems like a waste of computation/could make the classification worse. I could also replicate the same image across three channels, but I am similarly unsure of how it would preform.

keras-layer keras tensorflow
2 Answers

Keras pre-trained models have trained on color images and if you want to use their full power, you should use color images for fine-tuning. However, if you have grayscale images you can still use these pre-trained models by repeating your grayscale image over three channels. But obviously, it will not as well as using color images as input.

user_35409820

The VGG keras model uses the function: keras.applications.imagenet_utils._obtain_input_shape. This function was tailored for ImageNet data thus it enforces the input channel to be 3. One possible workaround will be to copy the VGG16 module and replace the line:

input_shape = _obtain_input_shape(input_shape, default_size=224, min_size=48, data_format=K.image_data_format(), include_top=include_top)

with:

input_shape = (im_height, im_width, 1)

As a side note, you will not be able to load ImageNet weights since your input space has changed and the first layer convolutions will not match.

user_39322120
Related