This code is provided by the official team of NeRF(2020) using tensorflow 1.14. I am trying to clone this model using tf.keras.clone_model(). The problem I am facing in the current code is that 'ReLU' object has no attribute '_name_'. The model is created with the following code. The summary of the model can be found at the end of the post.
def init_nerf_model(D=8, W=256, input_ch=3, input_ch_views=3, output_ch=4, skips=[4], use_viewdirs=False):
relu = tf.keras.layers.ReLU()
def dense(W, act=relu): return tf.keras.layers.Dense(W, activation=act)
print('MODEL', input_ch, input_ch_views, type(
input_ch), type(input_ch_views), use_viewdirs)
input_ch = int(input_ch)
input_ch_views = int(input_ch_views)
inputs = tf.keras.Input(shape=(input_ch + input_ch_views))
inputs_pts, inputs_views = tf.split(inputs, [input_ch, input_ch_views], -1)
inputs_pts.set_shape([None, input_ch])
inputs_views.set_shape([None, input_ch_views])
print(inputs.shape, inputs_pts.shape, inputs_views.shape)
outputs = inputs_pts
for i in range(D):
outputs = dense(W)(outputs)
if i in skips:
outputs = tf.concat([inputs_pts, outputs], -1)
if use_viewdirs:
alpha_out = dense(1, act=None)(outputs)
bottleneck = dense(256, act=None)(outputs)
inputs_viewdirs = tf.concat(
[bottleneck, inputs_views], -1) # concat viewdirs
outputs = inputs_viewdirs
# The supplement to the paper states there are 4 hidden layers here, but this is an error since
# the experiments were actually run with 1 hidden layer, so we will leave it as 1.
for i in range(1):
outputs = dense(W//2)(outputs)
outputs = dense(3, act=None)(outputs)
outputs = tf.concat([outputs, alpha_out], -1)
else:
outputs = dense(output_ch, act=None)(outputs)
model = tf.keras.Model(inputs=inputs, outputs=outputs)
return model
As I searched, the only solution I found was to create a sequential model like below, but I wonder how to make a sequential model of the same structure - like, how do you make the input layer and the concatenate layers? You can see the summary at the bottom.
model = Sequential()
model.add(Dense(W))
model.add(ReLU())
...
So I attempted three other solutions:
- Make an independent ReLU layer in the current construction style:
...
outputs = tf.keras.layers.Dense(W)(outputs)
outputs = tf.keras.layers.ReLU()(outputs)
- Use tf.keras.activations.relu instead. I tried to set it as the activation for Dense layer, and then as a separate layer, but neither worked. The error message was the same. \
- Copy the model with "=". However, when I change the new_grad_vars, the weights of the original model also change, which is not desired.
new_model = model
new_grad_vars = new_model.trainable_variables
Now, I'm looking for an answer of either sort:
a. How to create a clonable ReLU layer in the current way of construction
b. How to build the current model with input and concatenated layers as it is now using Sequential().
I would highly appreciate your help!
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 90)] 0
__________________________________________________________________________________________________
tf_op_layer_split (TensorFlowOp [(None, 63), (None, 0 input_1[0][0]
__________________________________________________________________________________________________
dense (Dense) (None, 256) 16384 tf_op_layer_split[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 256) 65792 dense[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 256) 65792 dense_1[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 256) 65792 dense_2[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 256) 65792 dense_3[0][0]
__________________________________________________________________________________________________
tf_op_layer_concat (TensorFlowO [(None, 319)] 0 tf_op_layer_split[0][0]
dense_4[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 256) 81920 tf_op_layer_concat[0][0]
__________________________________________________________________________________________________
dense_6 (Dense) (None, 256) 65792 dense_5[0][0]
__________________________________________________________________________________________________
dense_7 (Dense) (None, 256) 65792 dense_6[0][0]
__________________________________________________________________________________________________
dense_9 (Dense) (None, 256) 65792 dense_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_concat_1 (TensorFlo [(None, 283)] 0 dense_9[0][0]
tf_op_layer_split[0][1]
__________________________________________________________________________________________________
dense_10 (Dense) (None, 128) 36352 tf_op_layer_concat_1[0][0]
__________________________________________________________________________________________________
dense_11 (Dense) (None, 3) 387 dense_10[0][0]
__________________________________________________________________________________________________
dense_8 (Dense) (None, 1) 257 dense_7[0][0]
__________________________________________________________________________________________________
tf_op_layer_concat_2 (TensorFlo [(None, 4)] 0 dense_11[0][0]
dense_8[0][0]
==================================================================================================
Total params: 595,844
Trainable params: 595,844