Training the autoencoder with identical images

user_13109849 0
Viewed 168

I am training the autoencoder with 2000 identical images. My expectation is, that given the autoencoder has enough capacity the loss will approach 0 and the accuracy will approach 1 after a certain training time. Instead I see a quick convergence to loss = 0.07 and accuracy=0.76. Reducing the number of convolutional layers gave some improvement. Reducing the number of kernels per layer increased the loss. There is no improvement after that. Is my expectation wrong? Or is there something wrong with my autoencoder architecture? What can be done to make an almost lossless autoencoder?

input_img = Input(shape=(image_size_x, image_size_y, 1))
x = Conv2D(32, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((2, 2), padding='same')(x)
x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = MaxPooling2D((2, 2), padding='same')(x)

x = Conv2D(16, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(1, (3, 3), activation='sigmoid', padding='same')(x)

autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')

Thanks!

autoencoder keras tensorflow deep-learning
1 Answers

You need to add a dense layer between your autoconvolutional encoder and autoconvolution decoder. This is the latent reprensentation, also called embedding layer. This is the layer in which the image is compressed. That is the "compressed knowledge" that the architecture is trying to "learn".

For the implementation, from this tutorial: https://www.tensorflow.org/tutorials/generative/cvae I would suggest you add these lines between the encoder and the decoder part:

x = tf.keras.layers.Flatten()(x),
x = tf.keras.layers.Dense(latent_dim + latent_dim)
user_80313720
Related