Keras. Siamese network and triplet loss

user_11589183 0
Viewed 1035

I want to build a network that should be able to verificate images (e.g. human faces). As I understand, that the best solution for that is Siamese network with a triplet loss. I didn't found any ready-made implementations, so I decided to create my own.

But I have question about Keras. For example, here's the structure of the network:

network structure

And the code is something like that:

embedding = Sequential([
  Flatten(),
  Dense(1024, activation='relu'),
  Dense(64),
  Lambda(lambda x: K.l2_normalize(x, axis=-1))
])

input_a = Input(shape=shape, name='anchor')
input_p = Input(shape=shape, name='positive')
input_n = Input(shape=shape, name='negative')

emb_a = embedding(input_a)
emb_p = embedding(input_p)
emb_n = embedding(input_n)

out = Concatenate()([emb_a, emb_p, emp_n])

model = Model([input_a, input_p, input_n], out)

model.compile(optimizer='adam', loss=<triplet_loss>)

I defined only one embedding model. Does this mean that once the model starts training weights would be the same for each input?

If it is, how can I extract embedding weights from the model?

loss-function keras tensorflow computer-vision python
2 Answers

Yes, In triplet loss function weights should be shared across all three networks, i.e Anchor, Positive and Negetive. In Tensorflow 1.x to achieve weight sharing you can use reuse=True in tf.layers.

But in Tensorflow 2.x since the tf.layers has been moved to tf.keras.layers and reuse functionality has been removed. To achieve weight sharing you can write a custom layer that takes the parent layer and reuses its weights.

Below is the sample example to do the same.

class SharedConv(tf.keras.layers.Layer):
    def __init__(
        self,
        filters,
        kernel_size,
        strides=None,
        padding=None,
        dilation_rates=None,
        activation=None,
        use_bias=True,
        **kwargs
    ):
        self.filters = filters
        self.kernel_size = kernel_size
        self.strides = strides
        self.padding = padding
        self.dilation_rates = dilation_rates
        self.activation = activation
        self.use_bias = use_bias
        super().__init__(*args, **kwargs)

   def build(self, input_shape):
       self.conv = Conv2D(
           self.filters,
           self.kernel_size,
           padding=self.padding,
           dilation_rate=self.dilation_rates[0]
       )
       self.net1 = Activation(self.activation)
       self.net2 = Activation(self.activation)

   def call(self, inputs, **kwargs):
       x1 = self.conv(inputs)
       x1 = self.act1(x1)
       x2 = tf.nn.conv2d(
                inputs,
                self.conv.weights[0], 
                padding=self.padding,
                strides=self.strides,
                dilations=self.dilation_rates[1]
              )
        if self.use_bias:
            x2 = x2 + self.conv.weights[1]
        x2 = self.act2(x2)
        return x1, x2
user_142906970

I will answer on how to extract the embeddings (reference from my Github post):

My trained siamese model looked like this: siamese_model.summary()

image

Note that my newly redefined model is basically the same as the one highlighted in yellow

I then redefined my model which I wanted to use for extracting embeddings (It should be the same model you defined except now it will not have those multiple inputs like siamese) which looked like this:

siamese_embeddings_model = build_siamese_model(input_shape)

siamese_embeddings_model .summary()

image

Then I just extracted the weights from my trained siamese model and set them into my new model

embeddings_weights = siamese_model.layers[-3].get_weights()

siamese_embeddings_model.set_weights(embeddings_weights )

Then you can supply the new Image to extract the embeddings from the new model

vector = siamese.predict(image)

len(vector[0]) it will print 150 because of my fine dense layer (which are the output vector)

user_43237410
Related