3d input for Dense Layer Keras

Question

Is there any example of how Keras Dense layer handles 3D input.

The documentation explains the following:

If the input to the layer has a rank greater than 2, then Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 1 of the kernel (using tf.tensordot).

But I could not understand the internal matrix calculation

For example:

import tensorflow as tf
from tensorflow.keras.layers import Dense
sample_3d_input = tf.constant(tf.random.normal(shape=(4,3,2)))
dense_layer  = Dense(5)
op = dense_layer(sample_3d_input)

based on the documentation for a 3D input of shape (m,d0,d1), the shape of Layer's weight_matrix (or) kernel will have the shape (d1, units) which is (2,5) in this case. But I don't understand how the op is calculated to have the shape (m,d0, units)

user_10375049 · Answer

You can reproduce the operation on your own...

sample_3d_input = tf.constant(tf.random.normal(shape=(4,3,2)))
dense_layer  = Dense(5)

op = dense_layer(sample_3d_input)
W, b = dense_layer.get_weights()

manual_op = tf.tensordot(sample_3d_input, W, axes=[[2],[0]]) + b

Comparison:

>>> tf.reduce_all(op == manual_op).numpy()
... True

3d input for Dense Layer Keras

1 Answers