Huggingface load_dataset() method how to assign the "features" argument?

user_10054199 0
Viewed 743

I'm trying to load a custom dataset to use for finetuning a Huggingface model. My data is a csv file with 2 columns: one is 'sequence' which is a string , the other one is 'label' which is also a string, with 8 classes. I want to load my dataset and assign the type of the 'sequence' column to 'string' and the type of the 'label' column to 'ClassLabel'

my code is this:

from datasets import Features
from datasets import load_dataset


ft = Features({'sequence':'str','label':'ClassLabel'})

mydataset = load_dataset("csv", data_files="mydata.csv",features= ft)

running this code, I got the following error:

TypeError                                 Traceback (most recent call last)
<ipython-input-59-45fedff522e8> in <module>()
      7 
      8 mydataset = load_dataset("csv", data_files="mydata.csv",
----> 9                             features= ft)
     10 

8 frames
/usr/local/lib/python3.7/dist-packages/datasets/features/features.py in get_nested_type(schema)
    794 
    795     # Other objects are callable which returns their data type (ClassLabel, Array2D, Translation, Arrow datatype creation methods)
--> 796     return schema()

TypeError: 'str' object is not callable

could someone help please?

huggingface-transformers
1 Answers

You should define your Features similarly to what is in here:

from datasets import Features, Value, ClassLabel
from datasets import load_dataset

class_names = ['class_label_1', 'class_label_2']

ft = Features({'sequence': Value('string'), 'label': ClassLabel(names=class_names)})

mydataset = load_dataset("csv", data_files="mydata.csv",features=ft)
user_64636020
Related