How can I preprocess inputs sent to a hugging face estimator?

Viewed 8

I've been reviewing tutorials like the one found here which detail how to train a huggingface estimator (specifically, a transformer model) and then deploy it to sagemaker.

The tutorial linked above trains an estimator via:

huggingface_estimator = HuggingFace(entry_point='train.py',
                            source_dir='./scripts',
                            base_job_name='huggingface-sdk-extension',
                            instance_type='ml.p3.2xlarge',
                            instance_count=1,
                            transformers_version='4.4',
                            pytorch_version='1.6',
                            py_version='py37',
                            role=role,
                            hyperparameters = {'epochs': 1,
                                               'train_batch_size': 32,
                                               'model_name':'distilbert-base-uncased'
                                                })

Where train.py outlines a training script which tokenizes input data, and then fine-tunes a transformer model on the training data. Note: in this example, the training data is hard coded into train.py, but that is not the source of the issue I'm encountering.

The model is fit using

huggingface_estimator.fit()

and is then deployed using

predictor = huggingface_estimator.deploy(1,"ml.g4dn.xlarge")

But then this deployed model is used to make a prediction via:

sentiment_input= {"inputs" : "I love using the new Inference DLC."}
predictor.predict(sentiment_input)

The problem is that no one specifed anywhere how this input is to be preprocessed: the model does not work on raw text, the text must be tokenized. Even the official docs for sagemaker don't seem to outline how preprocessing for a deployed model is handled.

How can I specify a preprocessing step for my model?

0 Answers
Related