replicate a row tensor using tf.tile?

Viewed 27437

I have a tensor which is simply a vector, vector = [0.5 0.4] and tf.shape indicates that it has shape=(1,), I would like to replicate the vector m times and have the shape of [m, 2], so for m = 2, matrix = [[0.5 0.4], [0.5 0.4]]. How do I achieve that using tf.tile?

6 Answers

The same can be achieved by multiplying a ones matrix with vec and let broadcasting do the trick:

tf.ones([m, 1]) * vec

vec = tf.constant([1., 2., 3., 4.])
m = 3
matrix = tf.ones([m, 1]) * vec

with tf.Session() as sess:
   print(sess.run([matrix]))

#Output: [[1., 2., 3., 4.],
#         [1., 2., 3., 4.],
#         [1., 2., 3., 4.]]

replicating / duplicating a tensor (be it a 1D vector, 2D matrix, or any dimension) can be done by creating a list of copies of this tensor (with pure python), and then using tf.stack - having both steps in one (short) line. Here is an example of duplicating a 2D Tensor:

import tensorflow as tf
tf.enable_eager_execution()

a = tf.constant([[1,2,3],[4,5,6]])  # shape=(2,3)
a_stack = tf.stack([a] * 4)  # shape=(4,2,3)

print(a)
print(a_stack)

"[a]*4" creates a list containing four copies of the same tensor (this is pure python). tf.stack then stack them one after the other, on the first axis (axis=0)

In graph mode:

import tensorflow as tf

a = tf.constant([[1,2,3],[4,5,6]])  # shape=(2,3)
a_stack = tf.stack([a] * 4)  # shape=(4,2,3)

sess = tf.Session()
print('original tensor:')
print(sess.run(a))
print('stacked tensor:')
print(sess.run(a_stack))

An answer without reshaping:

vec = tf.constant([[1, 2, 3, 4]])
multiply = tf.constant([3, 1])
tf.tile(vec, multiply)

I assume that the main use case of such replication is to match the dimensionality of two tensors (that you want to multiply?).

In that case, there is a much simpler solution. Let the tensorflow do the work of dimensionality matching for you:

import tensorflow as tf
tf.enable_eager_execution()

a = tf.constant([1, 2, 3])  # shape=(3)
b = tf.constant([[[1, 3], [1, 3], [1, 3]], [[2, 0], [2, 0], [2, 0]]])  # shape=(2, 3, 2)

print(tf.einsum('ijk,j->ijk', b, a))

# OUTPUT:
# tf.Tensor(
# [[[1 3]
#   [2 6]
#   [3 9]]
# 
#  [[2 0]
#   [4 0]
#   [6 0]]], shape=(2, 3, 2), dtype=int32)

As you can see it can work for much more complex situations: when you need to replicate on both first and last dimensions, when you are working with more complex shapes, etc. All you need to do is match the indices in the string description (above we match the dimension of a, labeled j with the second dimension of b (ijk).

Another example use case: I have a state per neuron, and since we simulate in batches, this state has dimensionality (n_batch, n_neuron). I need to use this state to modulate connections between neurons (weights of synapses), which in my case had additional dimension so they have the dimensionality (n_neuron, n_neuron, n_X).

Instead of making a mess with tiling, reshaping, etc. I can just write it in a single line like so:

W_modulated = tf.einsum('ijk,bi->bijk', self.W, ux)

Tensorflow 2.0 solution: Refer to this link to read more about tf.tile

vector = [0.5, 0.4]
tf.reshape(tf.tile(vector, [2]), [2, tf.shape(vector)[0]])

# output 
<tf.Tensor: id=59, shape=(2, 2), dtype=float32, numpy=
array([[0.5, 0.4],
       [0.5, 0.4]], dtype=float32)>
Related