Tensorflow2.4 NotFoundError: No algorithm worked! with Keras Conv1D Layer

Question

I've been looking for a solution to this error for days and I can't find solutions for this:

NotFoundError: 3 root error(s) found.
  (0) Not found:  No algorithm worked!
 [[node model/conv1d/conv1d (defined at /lib/python3.6/threading.py:916) ]]
 [[div_no_nan/ReadVariableOp_1/_678]]
(1) Not found:  No algorithm worked!
 [[node model/conv1d/conv1d (defined at /lib/python3.6/threading.py:916) ]]
(2) Not found:  No algorithm worked!
 [[node model/conv1d/conv1d (defined at /lib/python3.6/threading.py:916) ]]
 [[Adam/concat_4/_704]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_152439]
Function call stack:
train_function -> train_function -> train_function

I'm trying to build my own model by combining BERT with a classifier like this but I am having some trouble implementing a Keras 1D ConvLayer in the classifier.

I'm using Tensorflow 2.4.1 and Python 3.6.9.

The output of nvcc --version is:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2020 NVIDIA Corporation
Built on Wed_Jul_22_19:09:09_PDT_2020
Cuda compilation tools, release 11.0, V11.0.221
Build cuda_11.0_bu.TC445_37.28845127_0

Any suggestions? This is my first question, so please notify me if more information is needed.

Minimal code to reproduce the error:

import numpy as np
import shutil
import tensorflow as tf
import tensorflow_hub as hub
import tensorflow_text as text

tfhub_handle_encoder = 'https://tfhub.dev/tensorflow/bert_en_uncased_L-12_H-768_A-12/3'
tfhub_handle_preprocess = 'https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3'

input_train = np.array([['this is such an amazing movie!'],
             ['Fat son how smiling mrs natural expense anxious friends. Boy scale enjoy ask abode fanny being son. As material in learning subjects so improved feelings'],
             ['Now indulgence dissimilar for his thoroughly has terminated. Agreement offending commanded my an. Change wholly say why eldest period.'],
             [' Are projection put celebrated particular unreserved joy unsatiable its. In then dare good am rose bred or. On am in nearer square wanted. ']            
            ])

y_train = np.array([0,0,1,0])

input_test = np.array([['Prevailed sincerity behaviour to so do principle mr. As departure at no propriety zealously my. On dear rent if girl view. First on smart there he sense.'],
             [' Delicate say and blessing ladyship exertion few margaret. Delight herself welcome against smiling its for. Suspected discovery by he affection household of principle perfectly he.'],
             ['In to am attended desirous raptures declared diverted confined at. Collected instantly remaining up certainly to necessary as.'],
             ['Over walk dull into son boy door went new. At or happiness commanded daughters as. Is handsome an declared at received in extended vicinity subjects.']            
            ])

y_val = np.array([1,0,1,1])


def build_classifier_model():
    text_input = tf.keras.layers.Input(shape=(), dtype=tf.string, name='text')
    preprocessing_layer = hub.KerasLayer(tfhub_handle_preprocess, name='preprocessing')
    encoder_inputs = preprocessing_layer(text_input)
    encoder = hub.KerasLayer(tfhub_handle_encoder, trainable=True, name='BERT_encoder')
    outputs = encoder(encoder_inputs)
    net = outputs['sequence_output']
    net = tf.keras.layers.Dropout(0.1)(net)
    net  = tf.keras.layers.Conv1D(512,5,activation='relu',strides=1)(net)
    net = tf.keras.layers.Dense(1, activation=None, name='classifier')(net)
    return tf.keras.Model(text_input, net)


strategy = tf.distribute.MirroredStrategy()

with strategy.scope():
    classifier_model = build_classifier_model()
    Adam = tf.keras.optimizers.Adam(lr=0.0002)
    classifier_model.compile(loss='sparse_categorical_crossentropy', optimizer=Adam, metrics=['accuracy'])
    history = classifier_model.fit(
        x = input_train,
        y= y_train,
        validation_data=(input_test, y_val),
        epochs=5,
        verbose=1,
    )

input_train, input_test, y_train and y_val are random values, but the error is the same.

user_11530462 · Answer

Please check after adding below code after importing tensorflow.Worked for me.

import tensorflow as tf
gpus = tf.config.experimental.list_physical_devices('GPU')
if gpus:
  # Restrict TensorFlow to only allocate 4GB of memory on the first GPU
  try:
    tf.config.experimental.set_virtual_device_configuration(
        gpus[0],
        [tf.config.experimental.VirtualDeviceConfiguration(memory_limit=4096)])
    logical_gpus = tf.config.experimental.list_logical_devices('GPU')
    print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
  except RuntimeError as e:
    # Virtual devices must be set before GPUs have been initialized
    print(e)
    `

Tensorflow2.4 NotFoundError: No algorithm worked! with Keras Conv1D Layer

1 Answers