RuntimeError: invalid argument 1:cannot perform reduction function max on tensor with no elements because the operation does not have an identity

Question

I am using a Pre-trained BERT model (FlauBERT) to produce features to give as input to multiple classifiers. In the example, they just show how to proceed for one signle sentence but I get a whole file (dataframe ) of approximately 40 000 sentences. So giving it all the model consume a lot of memory and I was looking for a way to pass small batch of sentences to the model so as to not created a killing of the system or "out of memory" error. I come up with a solution which is supposed to pass 2000 lines at at the time tp be process and then finally , I concatenate all the batch in one single numpy array. But for some reasons I get the following error :

layer : 
layer : 
layer : 
layer : 
layer : 
layer : 
layer : 
layer : 
layer : 
Traceback (most recent call last):
  File "knn_case_1.py", line 83, in 
    train_data_x_emb, mdl = get_flaubert_layer(train_data_x)
  File "knn_case_1.py", line 45, in get_flaubert_layer
    layer = flaubert(token_ids)
  File "/ho/geta/kelod/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/ho/geta/kelod/anaconda3/lib/python3.7/site-packages/transformers/modeling_flaubert.py", line 176, in forward
    assert lengths.max().item() <= slen
RuntimeError: invalid argument 1:cannot perform reduction function max on tensor with no elements because the operation does not have an identity

Here is the script : for reproductible environment : import transfromers ans scikit learn library

train_data = pd.read_csv('./corpus_train/corpus_ix_aug_FMC.csv', sep='	')
# read the train and test dataset
#train_data = pd.read_csv('./corpus_train/corpus_or_et_aug_avec_all_FM')
train_data_x = train_data.verbatim
train_data_y = train_data.etiquette

def spliterate(buf, chunk):
    for start in range(0, buf.size, chunk):
        yield buf[start:start + chunk]

def get_flaubert_layer(texte):
    
    modelname = "flaubert-base-cased"
    
    flaubert, log = FlaubertModel.from_pretrained(modelname, output_loading_info=True)
    flaubert_tokenizer = FlaubertTokenizer.from_pretrained(modelname, do_lowercase=False)
    tokenized = texte.apply((lambda x: flaubert_tokenizer.encode(x, add_special_tokens=True, max_length=512)))
    #tokenized = flaubert_tokenizer.encode(elt, add_special_tokens=True, max_length=512)
    max_len = 0
    for i in tokenized.values:
        if len(i) > max_len:
            max_len = len(i)
    padded = np.array([i + [0] * (max_len - len(i)) for i in tokenized.values])
    print(np.array(padded).shape)
    print(padded)
    attention_mask = np.where(padded != 0, 1, 0)
    print(attention_mask)
    print(attention_mask.shape)
    last_layer_ = []
    for tmp in spliterate(padded, 2000):
        if len(tmp) != 0:
            token_ids = torch.tensor(tmp)
            with torch.no_grad():
                layer = flaubert(token_ids)
                layer = layer [0][:,0,:].numpy()
                print("layer :", type(layer))
                last_layer_.append(layer)
            
            
                #last_layer += layer
        else:
            None 
    print("last layer_ :", type(last_layer_))
    print(len(last_layer_))     
    last_layer_np = numpy.stack(last_layer_, axis=0)
    print(len(last_layer_np))
    
    return last_layer_np, modelname 

train_data_x_emb, mdl = get_flaubert_layer(train_data_x)

padded variable look like this :

[[    1  1041 21565 ...     0     0     0]
 [    1   391   177 ...     0     0     0]
 [    1   150 14206 ...     0     0     0]
 ...
 [    1   150  5799 ...     0     0     0]
 [    1    59    48 ...     0     0     0]
 [    1   175    65 ...     0     0     0]]

RuntimeError: invalid argument 1:cannot perform reduction function max on tensor with no elements because the operation does not have an identity

0 Answers