Keras - One Class CNN - Two inputs one at each step

Question

I'm trying to train a One Class CNN but want only train on one class. So to solve the problem of not having a negative class instead of add random images to a 'no-class' label I want to train with noise for those inputs. But I want to input the noise no as a random generated image but as a features vector before the classifier NN, as it is explained at this paper: https://arxiv.org/pdf/1901.08688.pdf

So one step will come as a features vector from the VGG16 conv layers and the next step I would like to train with a random generated features vector.

I'm using tf.keras and the Functional API trying to add an extra input to VGG16. I've tried my own input generator, but I'm not sure if I can achieve it by code and using Keras since I don't know if I can activate different inpunts at each step. One approach is to extract all the features vectors from the conv layers, generate a similar number of noise vectors and then train the classificator with both the noise generated and the real features.

That will work I guess, but can I do it all in the same model?

Please, any help would be appreciated.

Thanks

user_4622046 · Answer

If you think slightly creatively, you don't need to have a noise input layer at all.

We can use GaussianNoise() from tf.keras, but it needs an input tensor, technically we should pass it a ones vector. We can make the intermediate VGG output features to multiply by zero kernels of a dense layer then we can add ones bias to it, so we get our placeholder ones vector which will be passed to GaussianNoise.

Now, you can forget any complex datagenerator, just use your regular data generators or directly numpy array with fit.

from tensorflow.keras.applications import VGG16
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
import tensorflow as tf

ip = Input((224,224,3))

base = VGG16((224,224,3))(ip)

# passing vgg features to a zero vector, making everything zero and then adding bias ones to make the output is always 1

dense_ones = Dense(1000, activation='linear', kernel_initializer = tf.keras.initializers.Zeros(), bias_initializer = tf.keras.initializers.Ones())(base)
gaussian = GaussianNoise(0.4)(dense_ones)

concat = Concatenate()([base, gaussian])

learn_feature = Dense(128, activation = 'relu')(concat) # change this part based on your needs
classification = Dense(2, activation = 'sigmoid')(learn_feature)

model = Model(ip, classification)

Model: "model_5"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_19 (InputLayer)           [(None, 224, 224, 3) 0                                            
__________________________________________________________________________________________________
vgg16 (Model)                   (None, 1000)         138357544   input_19[0][0]                   
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 1000)         1001000     vgg16[1][0]                      
__________________________________________________________________________________________________
gaussian_noise_5 (GaussianNoise (None, 1000)         0           dense_2[0][0]                    
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 2000)         0           vgg16[1][0]                      
                                                                 gaussian_noise_5[0][0]           
==================================================================================================
Total params: 139,358,544
Trainable params: 139,358,544
Non-trainable params: 0
___________________________________

Keras - One Class CNN - Two inputs one at each step

1 Answers