I'm trying to send a parameter as an environment variable for my deployed model. I ran a hyperparameter tuning run and I want to pass the string for one of the model parameters into the deployed endpoint.
I'm loading my trained PyTorch model with:
inference_model = PyTorchModel(
entry_point="inference.py",
source_dir="serve",
role=role,
model_data=model_data_s3_path,
env={'MODEL_ARCHITECTURE': best_architecture_name},
framework_version="1.8.1",
py_version="py36",
)
I'm using "env" to try and grab it during inference, but it doesn't seem to work when I use os.environ['MODEL_ARCHITECTURE']. I get an error that there is no such environment variable? What am I missing? Should I even be passing the string this way?
I'm also using os.environ['MODEL_ARCHITECTURE'] in the model.py script, which is called in inference.py.
How can I pass the string along? I need to do it this way (it needs to be automated, I can't change the model name by hand) since I'll be handing it off to some people to run it straight through and they can't manually change the architecture name.
EDIT: I tried SM_ARCHITECTURE_NAME since SageMaker often adds "SM_" to the environment variables / hyperparameters during training, but that didn't work either.