Keras model.predict with sigmoid activation and binary cross entropy returns only 0 or 1, not probability

user_9332174 0
Viewed 2939

I am building NN in Keras with overlapping class predictions. From what I understand, the 'sigmoid' activation function should return probability of belonging to a class. However, when I try to use model.predict(...), it just returns zeros or ones. Could you please tell me where I am going wrong and how I can get probabilitie of belonging to a class?

input_stock = Input(shape=input_shape,dtype='float32')  
stock_data= Conv2D(32, (1,5), padding='same', activation='relu')(input_stock)
stock_data = MaxPooling2D((1,2),strides=(1,2))(stock_data)
stock_data= Conv2D(8, (1,5), padding='same', activation='relu')(stock_data)
stock_data = MaxPooling2D((1,5),strides=(1,5))(stock_data)
stock_data= Conv2D(8, (154,5), padding='same', activation='relu')(stock_data)
stock_data = MaxPooling2D((1,2),strides=(1,2))(stock_data)

stock_data=Flatten()(stock_data)
stock_data=Dropout(.5)(stock_data)
output_layer = Dense(maxnumassets, activation='sigmoid')(stock_data)


model = Model(inputs=input_stock, outputs=output_layer)



return model


Output of model.predict (there are 150 classes), looks like below. 

array([[1., 1., 0., 0., 1., 1., 0., 1., 0., 0., 0., 1., 0., 1., 1., 1.,
        0., 0., 1., 1., 0., 0., 1., 0., 0., 1., 0., 0., 0., 0., 1., 0.,
        0., 1., 0., 0., 1., 1., 1., 1., 0., 1., 1., 0., 0., 0., 0., 1.,
        0., 1., 1., 1., 1., 1., 0., 0., 0., 0., 1., 1., 1., 0., 0., 1.,
        0., 0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 1., 1., 1.,
        1., 1., 0., 0., 1., 1., 1., 0., 0., 0., 1., 0., 1., 0., 0., 0.,
        1., 1., 1., 0., 0., 0., 1., 0., 1., 0., 1., 1., 1., 0., 1., 1.,
        0., 0., 1., 1., 1., 0., 0., 0., 0., 1., 1., 0., 1., 1., 1., 1.,
        0., 0., 0., 1., 1., 0., 0., 0., 1., 0., 1., 0., 1., 0., 1., 1.,
        1., 1., 0., 0., 0., 0.]], dtype=float32)'''

Dimensions of the data are as follows:

train (715, 150, 100, 2)
ytrain (715, 150)
xtest (80, 150, 100, 2)
ytest (80, 150)
keras-layer keras deep-learning machine-learning python
1 Answers

If you just use model.predict(xtest) directly, then your code will in general give values other than 0 and 1, cf. the example below which does nothing but apply your model to random data.

If yours doesn't, perhaps you're overfitting, or your model simply fits your data perfectly. Try applying model.predict to an array that has nothing to do with your data.

One other thing that seems off is that you've giving in your example output an array of shape (1, 150), but if xtest.shape is indeed (80, 150, 100, 2), then model.predict(xtest).shape is (80, 150).

input_shape = (150, 100, 2)
maxnumassets = 150
input_stock = Input(shape=input_shape,dtype='float32')
stock_data = Conv2D(32, (1,5), padding='same', activation='relu')(input_stock)
stock_data = MaxPooling2D((1,2),strides=(1,2))(stock_data)
stock_data = Conv2D(8, (1,5), padding='same', activation='relu')(stock_data)
stock_data = MaxPooling2D((1,5),strides=(1,5))(stock_data)
stock_data = Conv2D(8, (154,5), padding='same', activation='relu')(stock_data)
stock_data = MaxPooling2D((1,2),strides=(1,2))(stock_data)
stock_data = Flatten()(stock_data)
stock_data = Dropout(.5)(stock_data)
output_layer = Dense(maxnumassets, activation='sigmoid')(stock_data)

model = Model(inputs=input_stock, outputs=output_layer)
train = np.random.uniform(size=(775, 150, 100, 2))
ytrain = np.random.uniform(size=(775, 150))
xtest = np.random.uniform(size=(80, 150, 100, 2))
ytest = np.random.uniform(size=(80, 150))

model.compile('adam', 'mse')
model.fit(train, ytrain)
model.predict(xtest)

Output:

array([[0.50180054, 0.49852884, 0.49954456, ..., 0.4990623 , 0.49985167,
        0.4994816 ],
       [0.50180054, 0.49852884, 0.49954456, ..., 0.4990623 , 0.49985167,
        0.4994816 ],
       [0.50180054, 0.49852884, 0.49954456, ..., 0.4990623 , 0.49985167,
        0.4994816 ],
       ...,
       [0.50180054, 0.49852884, 0.49954456, ..., 0.4990623 , 0.49985167,
        0.4994816 ],
       [0.50180054, 0.49852884, 0.49954456, ..., 0.4990623 , 0.49985167,
        0.4994816 ],
       [0.50180054, 0.49852884, 0.49954456, ..., 0.4990623 , 0.49985167,
        0.4994816 ]], dtype=float32)
user_50852110
Related