I have this problem. I have two tensors, one shaped(batch_size=128, height=48, width = 48 , depth=1) that should contain indexes (from 0 to 32x32-1) and another one shaped (batch_size=128, height=32, width = 32 , depth=1) that contains the values that I should map. In this second each matrix in this batch contains its own values.
I would like to map for example, the third "index matrix" with the third "map matrix", considering that index inside each item of the batch range from 0 to 32x32. The same procedure should be applied to all the items in the batch. Since this stuff should be done in the loss function, and I see that we use batches there, how can I do this task? I thought that tf.gather could be helpful, since I've already used but in a simple case (like a constant array), but I don't know how to use it in this complex case.
Edited:
let's suppose I have:
[
[
[1,2,0,3],
[4,2,4,0],
[1,3,3,1],
[1,2,4,8]
],
[
[3,2,0,0],
[4,5,4,2],
[7,6,3,1],
[1,5,4,8]
]
] that is a (2,4,4,1) and a tensor
[
[
[0.3,0.4,0.6],
[0.9,0.2,0.5],
[0.1,0.2,0.1]
] ,
[
[0.1,0.4,0.5],
[0.8,0.1,0.6],
[0.2,0.4,0.3]
]
] that is a (2,3,3,1).
The first contains the indexes of the second.
I would like an output:
[
[
[0.4,0.6,0.3,0.9],
[0.2,0.6,0.2,0.3],
[0.4,0.9,0.9,0.4],
[0.4,0.6,0.2,0.1],
],
[
[0.8,0.5,0.1,0.1],
[0.1,0.6,0.1,0.5],
[0.4,0.2,0.8,0.4],
[0.4,0.6,0.1,0.3]
]
]
so the indexing should be referred to the single item of the batch. Should I also provide a derivative for this transformation?