Online Learning with Tensorflow

user_10287469 0
Viewed 4788

My problem is related to building a recommender system. I am using tensorflow to train the model with user_features, item_features and interactions.

How can i handle new data?

For example, whenever a use clicks on an item, i wanna record that interaction immediately and learn from that interaction (training the model for just that interaction). How can i do this?

Current implementation

Right now, i am following the traditional method of training the model first and then making predictions from that. In order to handle new interactions, i have to re-initialize the graph and retrain with all the data to make it work.

I hope there is some solution for this.

Thanks

reinforcement-learning tensorflow machine-learning python
2 Answers

You don't need to re-initialize the graph. Tensorflow is not explicitly aware of whether you're doing the training or testing phase when you request the output tensor. Rather, back propagation happens whenever you evaluate the optimizer's (e.g. tf.train.AdamOptimizer, tf.train.GradientDescentOptimizer, etc) minimize operation.

Normally you don't do this during testing, since you're trying to predict how well it performs on unseen data. But if you want to add that data to the training set after making the initial prediction, you're free to do so as long as you have a way to get the true value so it can compute the error tensor.

Even in non-online learning, it's common to intersperse training and testing passes: if you use a validation set to determine early stopping, you alternate between training batches, where you do do backprop, and validation batches, where you don't.

user_51960930

From what you are describing:

For example, whenever a use clicks on an item, i wanna record that interaction immediately and learn from that interaction (training the model for just that interaction). How can i do this?

What you might want to do is some sort of reinforcement learning, on top of your pretrained model.

The next thing to look into would be the OpenAI gym package.

A tutorial on how recommender systems can be brought together with reinforcement learning is given here in the first lecture (free to audit).

user_57635900
Related