I have a neural network in Keras that takes 23 inputs for 1 output in a regression task. I want to build a custom loss function, that has an extra term if some condition of the inputs is not fulfilled, i.e.:
loss_mse = tf.keras.losses.mean_squared_error(y_true, y_pred)
if y_pred > input23:
loss_dyn = (y_pred - input23)^2
loss = loss_mse + loss_dyn
so far I have tried to use basic python functions and all the TensorFlow equivalents (like tf.math.greater of tf.math.square) but both ways give errors. I used a wrapper function to feed x_train into the custom loss function as given below:
def my_loss_function_wrapper(names_dict, x):
def my_loss_function(y_true, y_pred):
loss_mse = tf.keras.losses.mean_squared_error(y_true, y_pred) # regluar MSE
def loss_dyn_true():
return tf.math.square(
tf.math.subtract(
y_pred,
x[:,23]
)
)
def loss_dyn_false():
return 0
loss_dyn = tf.cond(
tf.math.greater(
y_pred, x[22]
),
loss_dyn_true,
loss_dyn_false
)
def loss_stat_true():
return tf.math.square(
tf.math.subtract(
y_pred, x[21]
)
)
def loss_stat_false():
return 0
loss_stat = tf.cond(
tf.math.logical_and(
tf.math.less(
x[:,6]], 0.1
),
tf.math.less(
x[:,20]], 0.02
),
),
loss_stat_true,
loss_stat_false
)
loss = loss_mse + loss_dyn + loss_stat
return loss
return my_loss_function
but that gives the error
TypeError: true_fn and false_fn arguments to tf.cond must have the same number, type, and overall structure of return values. true_fn output: Tensor("Square:0", shape=(None, 9159), dtype=float32) false_fn output: Tensor("Const:0", shape=(), dtype=int32) Error details: Tensor("Square:0", shape=(None, 9159), dtype=float32) and Tensor("Const:0", shape=(), dtype=int32) have different types
9159 is the number if input samples for each feature, but what why is the Tensor("Const:0", shape=(), dtype=int32) of type integer, when it is my y_pred, that is supposed to be float?
And also, should I only pass the current sample of my input into the loss_function instead of the whole column of my input tensor (which has shape (None,9159) above)? How can I do that, sothat the according sample is accessed each time instead of the whole column.
Edit
Now I changed it to add the inputs that are relevant for my loss function to the output sothat I can access them as they are in y_true:
def my_loss_wrapper(names_dict):
def my_loss(y_true, y_pred):
loss_mse = tf.keras.losses.mean_squared_error(y_true, y_pred[:, 0]) # regluar MSE
def loss_dyn_true():
return tf.math.square(
tf.math.subtract(
y_pred[:, 0],
y_pred[:, names_dict['Sensor.IN01']]
)
)
def loss_dyn_false():
return y_pred[:, names_dict['Zeros']]
loss_dyn = tf.cond(
tf.math.greater(
y_pred[:, 0],
y_pred[:, names_dict['Car.02']]
),
loss_dyn_true,
loss_dyn_false
)
def loss_stat_true():
return tf.math.square(
tf.math.subtract(
y_pred[:, 0],
y_pred[:, names_dict['Car.02_stat']]
)
)
def loss_stat_false():
return y_pred[:, names_dict['Zeros']]
loss_stat = tf.cond(
tf.math.logical_and(
tf.math.less(
y_pred[:, names_dict['Sensor.02']], 0.1
),
tf.math.less(
y_pred[:, names_dict['Sensor.IN01']], 0.02
),
),
loss_stat_true,
loss_stat_false
)
loss = loss_mse + loss_dyn + loss_stat
return loss
return my_loss
(Names_dict points to the index of the corresponding value) which results in
ValueError: slice index 5 of dimension 1 out of bounds. for 'loss_7/output_1_loss/strided_slice_2' (op: 'StridedSlice') with input shapes: [?,1], [2], [2], [2] and with computed input tensors: input[1] = <0 5>, input[2] = <0 6>, input[3] = <1 1>.
It seems like the shapes are not in the right form.