Extracting last layers of keras model as a submodel

user_8480394 0
Viewed 12677

Say we have a convolutional neural network M. I can extract features from images by using 

extractor = Model(M.inputs, M.get_layer('last_conv').output)
features = extractor.predict(X)

How can I get the model that will predict classes using features?

I can't use the following lines because it requires the input of the model to be a placeholder.

predictor = Model([M.get_layer('next_layer').input], M.outputs)
pred = predictor.predict(features)

I also can't use K.function because later I want to use it as part of another model, so I will be appliyng predictor to tf.tensor, not np.array.

keras python
3 Answers

This is not the nicest solution, but it works:

from keras.models import Sequential
from keras.layers import Conv2D, MaxPooling2D
from keras.layers import Dense, Dropout, Flatten

def cnn():
    model = Sequential()
    model.add(Conv2D(32, kernel_size=(3, 3),
                     activation='relu',
                     input_shape=(28, 28, 1), name='l_01'))
    model.add(Conv2D(64, (3, 3), activation='relu', name='l_02'))
    model.add(MaxPooling2D(pool_size=(2, 2), name='l_03'))
    model.add(Dropout(0.25, name='l_04'))
    model.add(Flatten(name='l_05'))
    model.add(Dense(128, activation='relu', name='l_06'))
    model.add(Dropout(0.5, name='l_07'))
    model.add(Dense(10, activation='softmax', name='l_08'))
    return model

def predictor(input_shape):
    model = Sequential()
    model.add(Flatten(name='l_05', input_shape=(12, 12, 64)))
    model.add(Dense(128, activation='relu', name='l_06'))
    model.add(Dropout(0.5, name='l_07'))
    model.add(Dense(10, activation='softmax', name='l_08'))
    return model

cnn_model = cnn()
cnn_model.save('/tmp/cnn_model.h5')

predictor_model = predictor(cnn_model.output.shape)
predictor_model.load_weights('/tmp/cnn_model.h5', by_name=True)
user_43911290

Every layer in the model is indexed. So if you know which layers you need, you could loop through them, copying them into a new model. This operation should copy the weights inside the layer as well.

Here's a model (from Oli Blum's answer):

  model = Sequential()
  # add some layers
  model.add(Conv2D(32, kernel_size=(3, 3),
                 activation='relu',
                 input_shape=(28, 28, 1), name='l_01'))
  model.add(Conv2D(64, (3, 3), activation='relu', name='l_02'))
  model.add(MaxPooling2D(pool_size=(2, 2), name='l_03'))
  model.add(Dropout(0.25, name='l_04'))
  model.add(Flatten(name='l_05'))
  model.add(Dense(128, activation='relu', name='l_06'))
  model.add(Dropout(0.5, name='l_07'))
  model.add(Dense(10, activation='softmax', name='l_08'))

Say you wanted the last three layers:

def extract_layers(main_model, starting_layer_ix, ending_layer_ix):
  # create an empty model
  new_model = Sequential()
  for ix in range(starting_layer_ix, ending_layer_ix + 1):
    curr_layer = main_model.get_layer(index=ix)
    # copy this layer over to the new model
    new_model.add(curr_layer)
  return new_model
user_83350030

It depends on what you want to do.

  • If you are going to throw away the feature extractor afterwards
  • If you plan on training the feature extractor later

If you are going to use the extracted features but you don't intend on training the model used to generate them, you could use the predict method to get the features as you did:

features = extractor.predict(X)

then save its output to a file (np.save or cPickle or whatever). After that you could use that new dataset as the input to a new model.

If you plan on training the feature extractor later you'll need to stack the two networks as seen here with vgg as feature extractor https://github.com/fchollet/keras/issues/4576: 

img_width, img_height = 150, 150
vgg16_model = VGG16(include_top=False, weights='imagenet')

input = Input(batch_shape=vgg16_model.output_shape)
x = GlobalAveragePooling2D()(input)
x = Dense(256, activation='relu')(x)
x = Dropout(0.5)(x)
predict = Dense(1, activation='sigmoid')(x)
top_model = Model(input, predict)

top_model.load_weights(os.path.join(data_path, 'VGG16Classifier.hdf5'))  

input = Input(shape=(3, img_width, img_height))
x = vgg16_model(input)
predict = top_model(x)
model = Model(input, predict)

PS: This example uses channels first ordering. If you are using tensorflow you should change the shape to shape=(img_width, img_height,3 )

user_49559560
Related