I want to use a pre-trained Pytorch model in Tensorflow and I need to convert the tensorflow tensors to pytorch tensors. But I don't want to convert the pytorch tensor to a numpy array and convert that to a tensorflow tensor since I'm getting the error of " You must feed a value for placeholder tensor". I need this conversion when I'm making the graph so the tensorflow tensor doesn't have value and cannot be converted to numpy! Any solution for that?
1 Answers
Operations you do to Tensorflow tensors are "remembered" in order to calculate and back-propagate gradients. Same is true for PyTorch tensors. All this is ultimately required to train the model in both frameworks. This also is the reason why you can't convert tensors between the two frameworks: They have different ops and gradient calculation systems. They are incapable of capturing any operation that happens beyond their framework. For example, you can't (as of Jan 2021) have python for loops in custom loss functions. It has to be implemented into the framework in order to work. Similarly, there is no implementation of converting pytorch operations to Tensorflow operations.
This answer shows how it's done when your tensor is well-defined (not a placeholder). But there is currently no way to propagate gradients from Tensorflow to PyTorch or vice-versa. Maybe in the future there will be some kind of massive update to both frameworks that lets them inter-operate, but I doubt that. It's best to use them both separately.
So, in short, you can't convert placeholder tensors between two frameworks. You have to stick to one of the libraries or use concrete tensors + numpy mediator to communicate in-between frameworks.
Related