Tensorflow - TypeError: cannot pickle '_thread.lock' object

Viewed 114

I'm using Tensorflow 2.7.0 to build a very basic model with the following layers:

base_model = densenet.DenseNet121(weights='imagenet',
                                      include_top=False,
                                      input_shape=(224, 224, 3), pooling='avg')

predictions = tf.keras.layers.Dense(1,
                                        activation='sigmoid',
                                        name='predictions')(base_model.output)

_model = tf.keras.Model(inputs=base_model.input, outputs=predictions)

_model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-2),
                   loss='binary_crossentropy',
                   metrics=['accuracy'])

With the following callbacks:

csv_logger = tf.keras.callbacks.CSVLogger(csv_logger_path)

plateau = tf.keras.callbacks.ReduceLROnPlateau(monitor='val_loss', factor=0.1, patience=5)

db_logger = DAGsHubLogger(metrics_path=os.path.join(os.getcwd(), *model_const.METRICS_PATH),
                              hparams_path=os.path.join(os.getcwd(), *model_const.PARAMS_PATH))

model_checkpoint = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_filepath,
                                                          monitor='val_accuracy',
                                                          mode='max',
                                                          save_best_only=True)

However when I run it the following message appears and the terminal stops responding: TypeError: cannot pickle '_thread.lock' object

I've checked for this issue and found this issue and this question but as far as I understand the solutions there can not be relevant to my model.

Would very much appreciate any lead.

1 Answers

With the gpu loading of the model, the gpu ram may have overflowed. Can you try a lighter model like Resnet34.

altenative check this comment

Related