Is it possible to mask Neural Network output variables depending on input variables

user_17569213 0
Viewed 135

I have a weird use case for a neural network and want to understand if there is a way to accomplish what I'm trying to do.

I am trying to train a neural network that takes in 3 input variables and outputs 96 continuous variables. The output should ideally produce a continuous curve, however the expected y values have a lot of missing data points (>50%) distributed randomly which affects how the model trains. I know which data points are missing and am trying to find a way to ignore these outputs during backpropagation.

For example:

Input = [1,2,3]
Expected Output = [1,2,3,NAN,5,6,7,NAN,...] # NAN is set to 0 for training

Currently this is the method I am trying (tensorflow.keras)

in1 = layers.Input(3)
in2 = layers.Input(96) # Array of Bools, =1 if expected output variable is a number, =0 if nan
hidden1 = layers.Dense(37,activation='relu',use_bias=True)(in1)
hidden2 = layers.Dense(37,activation='relu',use_bias=True)(hidden1)
hidden3 = layers.Dense(37,activation='relu',use_bias=True)(hidden2)
hidden3_in2 = layers.concatenate([hidden3,in2])
out = layers.Dense(96)(hidden3_in2)

model = Model(inputs=[in1,in2], outputs=[out])

The expected output of this should be 0 being calculated where in2 == 0, and a number greater than 0 everywhere else. When using the model to predict data I plug in an array of 1's into in2, indicating that no expected values should equal 0, so a continuous curve should be output. However, many output variables still come out to 0, which is not ideal.

Essentially my question is: is there a good way to mask specific outputs during backprop and/or loss calculation using an array?

Thanks in advance!

keras tensorflow deep-learning neural-network python
1 Answers

All you have to do is to write your custom loss function, where you literally mask out the loss values.

Something along the lines of

def my_loss_fn(y_true, y_pred):
    squared_difference = tf.square(y_true - y_pred)
    mask = 1.0 - tf.math.is_nan(y_true)
    return tf.reduce_mean(squared_difference * mask, axis=-1)  # Note the `axis=-1`

model.compile(optimizer='adam', loss=my_loss_fn)
user_26580500
Related