TorchServe MAR multiple Python files per model

user_4515492 0
Viewed 946

Is there a way to split a model between multiple files before calling MAR?

For example, if I have the file base_model.py:

import torch
from torch import nn


class BaseModel(nn.Module):
    ...

And a model that inherits from BaseModel:

from base_model import BaseModel


class FullModel(BaseModel):
    ...

The same goes for the handler. I don't know how to split the code between multiple files.

pytorch python
1 Answers

Short answer: Yes

You can pass main model file and remaining files as extra-files. For example: full_model.py using --model-file ./xxx/aaa/full_model.py and all the other files using --extra-files ./xxx/bbb/base_model_1.py,./xxx/ccc/base_model_2.py,./xxx/ddd/mini_handler.py,./xxx/eee/mini_handler_2.py. All the files will be in a single folder before packaging as .mar file. With this in mind you might need to change imports in your python files a bit. All in all, it should be similar to

torch-model-archiver \
    --model-name recommendation \
    --version 1.0 \
    --model-file src/model/full_model.py \
    --serialized-file model/model.pt \
    --handle model_handler.py \
    --extra-files \
"dd/utils/utils.py,\
ee/network_config.json,\
ff/utils/preprocess.py,\
gg/utils/data_loader.py\

Now you can use functions from preprocess or dataloader in model handler by normal imports

user_70354480
Related