Dynamically Feeding the output of an RNN as input in next time step during training

I want to achieve the following functionality during "training" I feed my RNN with an input and get an output for that time step. I then take that output and feed it again to my RNN as input for the next time step and so on for say next 10 steps. After these 10 steps, I need to backpropagate for this sequence(of 10 inputs). From my knowledge, tensorflow requires feeding the entire sequence for training unlike my case where input is getting generated during the run itself. So is that possible to achieve in tensorflow?