With BERT Text Classification, ValueError: too many dimensions 'str' error occuring

user_14952121 0
Viewed 17091

Trying to make a classifier for sentiments of texts with BERT model but getting ValueError : too many dimensions 'str'

That is the DataFrame for values of train data; so they are train_labels

0   notr
1   notr
2   notr
3   negative
4   notr
... ...
854 positive
855 notr
856 notr
857 notr
858 positive

and there is the code which is producing the error for

train_seq = torch.tensor(tokens_train['input_ids'])
train_mask = torch.tensor(tokens_train['attention_mask'])
train_y = torch.tensor(train_labels.tolist())

At train_y = torch.tensor(train_labels.tolist()); getting error: ValueError: too many dimensions 'str'

can you help me pleaseenter image description here

enter image description here

mlp text-classification bert-language-model tensor python
4 Answers

I had the same problem: This worksfor me I guess you need to do it at the beginning of your code after reading csv: df['labels'] = df['labels'].replace(['negative','notr','positive'],[0,1,2])

then split for training and testing from these labels.

user_37646520

REASON

The issue is you are passing a list of strings (str) in torch.tensor() , it only accepts the list of numerical values (integer, float etc.) .

SOLUTION

So I suggest you to convert your string labels into integer values before passing it to the torch.tensor().

IMPLEMENTATION

Following code might help you

# a temporary list to store the string labels
temp_list = train_labels.tolist()

# dictionary that maps integer to its string value 
label_dict = {}

# list to store integer labels 
int_labels = []

for i in range(len(temp_list)):
    label_dict[i] = temp_list[i]
    int_labels.append(i)

Now pass this int_labels to the torch.tensor and use it as label.

train_y = torch.tensor(int_labels)

and whenever you want to see the respective string label of any integer just use label_dict dictionary.

user_78501740
user_54621000

Thanks, it did converting to integer, but there is a problem about classification;

0
0   positive
1   negative
2   positive
3   notr
4   positive
... ...
4002    notr
4003    positive
4004    positive
4005    notr
4006    negative

Frame had that datas, after the convert to int,

0   0
1   1
2   2
3   3
4   4
... ...
4002    4002
4003    4003
4004    4004
4005    4005
4006    4006

it become like that, what I need is all positives , neutrals and negatives representing as 0 for neg-1 for neutral - 2 for pos

user_149521210
Related