Weird accuracy in multilabel classification keras

Viewed 3186

I have a multilabel classification problem, I used the following code but the validation accuracy jumps to 99% in the first epoch which is weird given the complexity of the data as the input features are 2048 extracted from inception model (pool3:0) layer and the labels are [1000],(here is the link of a file contains samples of features and label : https://drive.google.com/file/d/0BxI_8PO3YBPPYkp6dHlGeExpS1k/view?usp=sharing ), is there something I am doing wrong here ??

Note: labels are sparse vector contain only 1 ~ 10 entry as 1 the rest is zeros

model.compile(optimizer='adadelta', loss='binary_crossentropy', metrics=['accuracy']) 

The output of prediction is zeros !

What wrong I do in training the model to bother the prediction ?

#input is the features file and labels file

def generate_arrays_from_file(path ,batch_size=100):
x=np.empty([batch_size,2048])
y=np.empty([batch_size,1000])
while True:
    f = open(path)
    i = 1  
    for line in f:
        # create Numpy arrays of input data
        # and labels, from each line in the file
        words=line.split(',')
        words=map(float, words[1:])
        x_= np.array(words[0:2048])
        y_=words[2048:]
        y_= np.array(map(int,y_))
        x_=x_.reshape((1, -1))
        #print np.squeeze(x_)
        y_=y_.reshape((1,-1))
        x[i]= x_
        y[i]=y_
        i += 1
        if i == batch_size:
            i=1
            yield (x, y)

    f.close()

model = Sequential()
model.add(Dense(units=2048, activation='sigmoid', input_dim=2048))
model.add(Dense(units=1000, activation="sigmoid", 
kernel_initializer="uniform"))
model.compile(optimizer='adadelta', loss='binary_crossentropy', metrics=
['accuracy'])

model.fit_generator(generate_arrays_from_file('train.txt'),
                validation_data= generate_arrays_from_file('test.txt'),
                validation_steps=1000,epochs=100,steps_per_epoch=1000, 
                  verbose=1)
2 Answers

Did you try to use the cosine similarity as loss function?

I had the same multi-label + high dimensionality problem.

The cosine distance takes account of the orientation of the model output (prediction) and the desired output (true class) vector.

It is the normalized dot-product between two vectors.

In keras the cosine_proximity function is -1*cosine_distance. Meaning that -1 corresponds to two vectors with the same size and orientation.

Related