I'm currently trying to take Adam's second order moment term, v_t, and use that as an additional term in my cost function. How can I implement something like this:
Cost = Cross Entropy + v_t*some_function(weights)
Can this be accomplished within python? Or do I have to write my own C++ code to accomplish this? Also is this easily accomplished in a framework like Keras? Here's the code for the cost function that I'm trying to add into keras:
def my_loss(y_pred, y_true, current_weights, v_t):
normal_loss=K.categorial_cross_entropy(y_pred,y_true)
additional_term=K.dot(K.square(current_weights - K.some_function(current_weights)), v_t)
return normal_loss + additional_term