Extremely low loss on training, prediction is always the same value

user_13561091 0
Viewed 122

Issue summary

Training my data results in very strange verbose:

loss: -35657827.5218 - accuracy: 0.5018 - categorical_accuracy: 1.0000

Expected prediction result is -1, 0 or 1. But the trained model only outputs 0

Data

I prepared two million rows of data with the following structure. These are all the features. They are float values between -1 and 1.

[
    "relative_price_week",
    "relative_price_day",
    "relative_price_hour",
    "relative_price_fivemin",
    "relative_volume_week",
    "relative_volume_day",
    "relative_volume_hour",
    "swing_month",
    "swing_week",
    "swing_day",
    "swing_hour",
    "swing_fivemin",
]

And the output is expected to be one of three options, -1, 0 or 1.

Visualized data

Model

input = Input(shape=(12,))
x = Dense(64, activation='relu')(input)
x = Dense(32, activation='relu')(x)
x = Dense(16, activation='relu')(x)
output = Dense(1, activation='sigmoid')(x)
model = Model(inputs=input, outputs=output)
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', 'categorical_accuracy'])
model.fit(train_features, train_labels, batch_size=100000, epochs=10)

Previous versions of the model would also result in nonsensical data. By which i mean, it would output only one of several fixed outputs.
One version would have three output columns ['buy_conf', 'hold_conf', 'sell_conf'], each representing a value between 0 and 1.
Unfortunately, this too would result in strange outputs with accuracy being 0.9 and a similar negative loss. The trained output for this model would also put one of very few options as the output for each column.

Questions

  • Do you see anything immediately obviously wrong with this code?
  • Can you suggest an approach to find a way of solving this issue?
tf.keras tensorflow python
2 Answers

sigmoid activation function can output a value between 0 & 1
But you have labels -1,0,1 which is not in the range 0 & 1
Therefore, Using tanh activation function in the final Dense layer instead of sigmoid will work as tanh has a range of -1 to 1

user_157606240

I didn't end up finding a solution using Tensorflow.
I ended up using ML.net instead which works better for my situation since the rest of my application uses C#.
It was also beginner-friendly and make it easy to do exactly what i wanted to do.

This isn't the solution for everyone but if someone else reads this, maybe it's useful in your situation too.

user_135610910
Related