I am using arabic Bert and passing my training datasets as batches, the error message that I get is that Tensor can not be treated as numpy array but could not detach to numpy using detach().numpy()
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
<ipython-input-9-271b41658d5b>:46 call *
x = self.embed_with_bert(inputs)
<ipython-input-9-271b41658d5b>:39 embed_with_bert *
embds = self.bert_layer(all_tokens[:,0,:].detach().numpy(),
AttributeError: 'Tensor' object has no attribute 'detach'
also when I change the tokenizer to 'tf'
I get the following error:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
<ipython-input-17-b2127b5212bc>:46 call *
x = self.embed_with_bert(inputs)
<ipython-input-53-b99c90611f94>:39 embed_with_bert *
embds = self.bert_layer([all_tokens[:,0,:],
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py:550 __call__ *
result = self.forward(*input, **kwargs)
/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py:691 forward *
input_shape = input_ids.size()
AttributeError: 'list' object has no attribute 'size'
tried this also
embds = self.bert_layer(tf.unstack(all_tokens[:,0,:]),
tf.unstack(all_tokens[:,1,:]),
tf.unstack(all_tokens[:,2,:]))
but it generated:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py:571 train_function *
outputs = self.distribute_strategy.run(
<ipython-input-64-e1ff853b33fb>:55 call *
x = self.embed_with_bert(inputs)
<ipython-input-29-d3665857b399>:44 embed_with_bert *
embds = self.bert_layer(tf.unstack(all_tokens[:,0,:]),tf.unstack(all_tokens[:,1,:]),tf.unstack(all_tokens[:,2,:])) #[:,0,:],all_tokens[:,1,:],all_tokens[:,2,:])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/array_ops.py:1510 unstack **
raise ValueError("Cannot infer num from shape %s" % value_shape)
ValueError: Cannot infer num from shape (None, None)
Example: --> all_tokens are tensor sent as batch to the model but now I want to use it as numpy array to fix the error?
embds = self.bert_layer(all_tokens[:,0,:],
all_tokens[:,1,:],
all_tokens[:,2,:])
though:
this work fine [t is generated from the tokenizer of hugging-face:
a = bert_layer(t["input_ids"], t["attention_mask"], t["token_type_ids"])