I've been reviewing tutorials like the one found here which detail how to train a huggingface estimator (specifically, a transformer model) and then deploy it to sagemaker.
The tutorial linked above trains an estimator via:
huggingface_estimator = HuggingFace(entry_point='train.py',
source_dir='./scripts',
base_job_name='huggingface-sdk-extension',
instance_type='ml.p3.2xlarge',
instance_count=1,
transformers_version='4.4',
pytorch_version='1.6',
py_version='py37',
role=role,
hyperparameters = {'epochs': 1,
'train_batch_size': 32,
'model_name':'distilbert-base-uncased'
})
Where train.py outlines a training script which tokenizes input data, and then fine-tunes a transformer model on the training data. Note: in this example, the training data is hard coded into train.py, but that is not the source of the issue I'm encountering.
The model is fit using
huggingface_estimator.fit()
and is then deployed using
predictor = huggingface_estimator.deploy(1,"ml.g4dn.xlarge")
But then this deployed model is used to make a prediction via:
sentiment_input= {"inputs" : "I love using the new Inference DLC."}
predictor.predict(sentiment_input)
The problem is that no one specifed anywhere how this input is to be preprocessed: the model does not work on raw text, the text must be tokenized. Even the official docs for sagemaker don't seem to outline how preprocessing for a deployed model is handled.
How can I specify a preprocessing step for my model?