I'm currently working on a neural network which is supposed to adjust a camera pose based on a given pose and an image.
I can't get the network to train and my question is - what am I doing wrong ?
Data gets loaded by a custom dataloader function. The network and training setup script:
import dataloader
from tensorflow.keras import optimizers, Model, Input
from tensorflow.keras.callbacks import TensorBoard, ModelCheckpoint
from tensorflow.keras.models import Sequential, load_model
from tensorflow.keras.layers import Dense, concatenate
tensorboard_callback = TensorBoard(
log_dir="./tensorboard_logs",
histogram_freq=1,
write_graph=True,
write_images=False,
update_freq="epoch",
)
cp_callback = ModelCheckpoint(filepath="./model",
monitor="loss",
save_best_only=True,
save_weights_only=False,
verbose=1)
first_input = Input(shape=(25088,))
first_dense = Dense(6000, activation="relu")
x = first_dense(first_input)
x = Dense(1500, activation="relu")(x)
x = Dense(375, activation="relu")(x)
second_input = Input(shape=(6,))
merged = concatenate([x, second_input])
output = Dense(6, activation="sigmoid")(merged)
model = Model(inputs=[first_input, second_input], outputs=output, name="network_name")
# define optimizer
opt = optimizers.Adam(
learning_rate=0.001,
beta_1=0.9,
beta_2=0.999,
epsilon=1e-07,
amsgrad=False,
name="Adam"
)
# Compile the model with optimizers and loss functions
model.compile(optimizer=opt, loss="MeanSquaredError", metrics=["MeanSquaredError"])
model.summary()
train_gen = dataloader.Dataloader().dataloader(batch_size=5, training=True)
val_gen = dataloader.Dataloader().dataloader(batch_size=5, training=True)
# Train model
model.fit(
x=train_gen,
validation_data=next(val_gen),
# validation_data=tuple(train_gen),
validation_steps=1,
steps_per_epoch=7,
initial_epoch=1,
epochs=1000,
shuffle=False,
callbacks=[tensorboard_callback, cp_callback]
)
model.save("./model/new_model.h5")
This results in following architecture:
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 25088)] 0
__________________________________________________________________________________________________
dense (Dense) (None, 6000) 150534000 input_1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 1500) 9001500 dense[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 375) 562875 dense_1[0][0]
__________________________________________________________________________________________________
input_2 (InputLayer) [(None, 6)] 0
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 381) 0 dense_2[0][0]
input_2[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 6) 2292 concatenate[0][0]
==================================================================================================
Total params: 160,100,667
Trainable params: 160,100,667
Non-trainable params: 0
__________________________________________________________________________________________________
The first input is the mentioned image. the second input is the to be adjusted pose.
My issue is that there seems to be an error in how I hand over the data from my dataloader to the fit() function. I receive following error:
2022-02-03 13:17:42.111511: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 2/1000
Traceback (most recent call last):
File "C:\path\network.py", line 73, in <module>
model.fit(
File "C:\Python39\lib\site-packages\keras\engine\training.py", line 1184, in fit
tmp_logs = self.train_function(iterator)
File "C:\Python39\lib\site-packages\tensorflow\python\eager\def_function.py", line 885, in __call__
result = self._call(*args, **kwds)
File "C:\Python39\lib\site-packages\tensorflow\python\eager\def_function.py", line 933, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "C:\Python39\lib\site-packages\tensorflow\python\eager\def_function.py", line 759, in _initialize
self._stateful_fn._get_concrete_function_internal_garbage_collected( # pylint: disable=protected-access
File "C:\Python39\lib\site-packages\tensorflow\python\eager\function.py", line 3066, in _get_concrete_function_internal_garbage_collected
graph_function, _ = self._maybe_define_function(args, kwargs)
File "C:\Python39\lib\site-packages\tensorflow\python\eager\function.py", line 3463, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "C:\Python39\lib\site-packages\tensorflow\python\eager\function.py", line 3298, in _create_graph_function
func_graph_module.func_graph_from_py_func(
File "C:\Python39\lib\site-packages\tensorflow\python\framework\func_graph.py", line 1007, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "C:\Python39\lib\site-packages\tensorflow\python\eager\def_function.py", line 668, in wrapped_fn
out = weak_wrapped_fn().__wrapped__(*args, **kwds)
File "C:\Python39\lib\site-packages\tensorflow\python\framework\func_graph.py", line 994, in wrapper
raise e.ag_error_metadata.to_exception(e)
ValueError: in user code:
C:\Python39\lib\site-packages\keras\engine\training.py:853 train_function *
return step_function(self, iterator)
C:\Python39\lib\site-packages\keras\engine\training.py:842 step_function **
outputs = model.distribute_strategy.run(run_step, args=(data,))
C:\Python39\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:1286 run
return self._extended.call_for_each_replica(fn, args=args, kwargs=kwargs)
C:\Python39\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:2849 call_for_each_replica
return self._call_for_each_replica(fn, args, kwargs)
C:\Python39\lib\site-packages\tensorflow\python\distribute\distribute_lib.py:3632 _call_for_each_replica
return fn(*args, **kwargs)
C:\Python39\lib\site-packages\keras\engine\training.py:835 run_step **
outputs = model.train_step(data)
C:\Python39\lib\site-packages\keras\engine\training.py:787 train_step
y_pred = self(x, training=True)
C:\Python39\lib\site-packages\keras\engine\base_layer.py:1020 __call__
input_spec.assert_input_compatibility(self.input_spec, inputs, self.name)
C:\Python39\lib\site-packages\keras\engine\input_spec.py:199 assert_input_compatibility
raise ValueError('Layer ' + layer_name + ' expects ' +
ValueError: Layer network_name expects 2 input(s), but it received 10 input tensors. Inputs received: [<tf.Tensor 'ExpandDims:0' shape=(None,
1) dtype=float32>, <tf.Tensor 'ExpandDims_1:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_2:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_3:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_4:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_5:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_6:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_7:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_8:0' shape=(None, 1) dtype=float32>, <tf.Tensor 'ExpandDims_9:0' shape=(None, 1) dtype=float32>]
I found an article (sadly can't find it anymore and didn't save the link) which said that the problem could be solved by applying the tuple() function to val_generator in the fit() function, but that resulted in my script just doing nothing after compiling the network
Please also let me know if there are general problems with the network architecture or how to improve. Since this is my first real neural network project I'm not really certain about anything I'm doing to be honest.
If there is any additional info, one might need- please let me know.
Thanks in advance