'Tensor' object has no attribute 'detach'

Viewed 5610

I am using arabic Bert and passing my training datasets as batches, the error message that I get is that Tensor can not be treated as numpy array but could not detach to numpy using detach().numpy()

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
    outputs = self.distribute_strategy.run(
<ipython-input-9-271b41658d5b>:46 call  *
    x = self.embed_with_bert(inputs)
<ipython-input-9-271b41658d5b>:39 embed_with_bert  *
    embds = self.bert_layer(all_tokens[:,0,:].detach().numpy(),

AttributeError: 'Tensor' object has no attribute 'detach'

also when I change the tokenizer to 'tf'

I get the following error:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
    outputs = self.distribute_strategy.run(
<ipython-input-17-b2127b5212bc>:46 call  *
    x = self.embed_with_bert(inputs)
<ipython-input-53-b99c90611f94>:39 embed_with_bert  *
    embds = self.bert_layer([all_tokens[:,0,:],
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:550 __call__  *
    result = self.forward(*input, **kwargs)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py:691 forward  *
    input_shape = input_ids.size()

AttributeError: 'list' object has no attribute 'size'

tried this also

embds = self.bert_layer(tf.unstack(all_tokens[:,0,:]),
                            tf.unstack(all_tokens[:,1,:]),
                            tf.unstack(all_tokens[:,2,:])) 

but it generated:

/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function  *
    outputs = self.distribute_strategy.run(
<ipython-input-64-e1ff853b33fb>:55 call  *
    x = self.embed_with_bert(inputs)
<ipython-input-29-d3665857b399>:44 embed_with_bert  *
    embds = self.bert_layer(tf.unstack(all_tokens[:,0,:]),tf.unstack(all_tokens[:,1,:]),tf.unstack(all_tokens[:,2,:])) #[:,0,:],all_tokens[:,1,:],all_tokens[:,2,:])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:1510 unstack  **
    raise ValueError("Cannot infer num from shape %s" % value_shape)

ValueError: Cannot infer num from shape (None, None)

Example: --> all_tokens are tensor sent as batch to the model but now I want to use it as numpy array to fix the error?

embds = self.bert_layer(all_tokens[:,0,:],
                               all_tokens[:,1,:],
                               all_tokens[:,2,:])

though:

this work fine [t is generated from the tokenizer of hugging-face:

a = bert_layer(t["input_ids"], t["attention_mask"], t["token_type_ids"])
0 Answers
Related