How to log custom models in mlflow inside artifacts of a run?

user_14087348 0
Viewed 3660

I've stored my model in Github with MLProject so that others may run it in various variations. Now I would like to log the models created by runs as artifacts so that users can load them with MLModel. As my model is custom this is not as simple as with flavors such as spark. I first saved and loaded the model locally to make sure my Environment, model and artifact Code works. Now I want to include the Login of the model as part of the MLProject run from GitHub. As there is no example in the documentation I know of, I wanted to ask for help and suggest that this may be a good addition to the documentation or examples.

In Terms of Code I wrote the follwing inside the mlrun at the end:

Model depending on the Parameters specified in the Project run

 ETS_Exogen = ETS_Exogen(params=res.x, before=before,after=after)

logging the model with the previously defined model, evironment and artifacts

mlflow.pyfunc.log_model(python_model=ETS_Exogen, conda_env=conda_env,artifacts=artifacts)

Does mlflow.pyfunc.log_model automatically log the model into the artifacts of the runs or do I Need to define an artifact_path? Should I rather use mlflow.pyfunc.save_model ? I defined the artifacts paths so that they would be gathered from the GitHub repository as follows:

artifacts = { "exogen_variables":os.path.join(os.path.dirname(os.path.abspath(file)),"exogen_variables.csv") }

Is this correct? Links to Documentation on custom Models: https://mlflow.org/docs/latest/python_api/mlflow.pyfunc.html#pyfunc-create-custom-workflows

mlflow python
1 Answers

logging a model Needs a path, Standard is to store it in artifacts under the Folder models. The command is as follows:

mlflow.pyfunc.log_model(artifact_path="model",python_model=ETS_Exogen, conda_env=conda_env)

Here is how to add data in the model from a http Server. Dont use artifact but rather load it directly with Pandas in the context.

def load_context(self, context):
    import numpy as np
    import pandas as pd #data wrangeling
    url_to_exogen_raw = 'https://raw.githubusercontent.com/MatthiasHerp/ETS_Ex_BA_MLFlow/master/exogen_variables.csv'
    self.exogen = pd.read_csv(url_to_exogen_raw, index_col='date')
user_140873480
Related