I am trying to save runs called from MLflow Projects to specific experiment names/ids.
Currently, my MLflow project looks like this:
name: Echo NLP Project
entry_points:
generate:
parameters:
...
command: "python scripts/generate_data.py ... --exp_name {exp_name}"
The project calls in the command to the script generate_data.py where I would do some functions and record a run in the function.
I would like to specify where this run would be saved with the exp_name parameter and I am doing this by having this code in generate_data.py:
mlflow.set_experiment(experiment_name)
mlflow.run(uri=os.getcwd(), experiment_name=experiment_name)
However, it is giving me this error:
2021/11/12 15:45:06 INFO mlflow.projects.utils: === Created directory /tmp/tmp3jx68uph for downloading remote URIs passed to arguments of type 'path' ===
2021/11/12 15:45:06 INFO mlflow.projects.backend.local: === Running command 'source /.../profile.d/conda.sh && conda activate mlflow-667a7951db8b0b02e9ac567ea087405cbc28af02 1>&2 && python scripts/generate_data.py ... --exp_name test' in run with ID 'cd2b5b20f00344e5a6257deafabc4791' ===
Traceback (most recent call last):
File "scripts/generate_data.py", line 32, in <module>
x = tm(json_folder=args.json_folder, spacy_folder=args.spacy_folder, report_source=args.reports, experiment_name=args.exp_name)
File "/mnt/c/Users/username/Documents/echo-nlp-model/scripts/train_class.py", line 157, in __init__
mlflow.run(uri=os.getcwd(), experiment_name=experiment_name)
File "/mnt/c/Users/username/Documents/venv/lib/python3.8/site-packages/mlflow/projects/__init__.py", line 293, in run
submitted_run_obj = _run(
File "/mnt/c/Users/username/Documents/venv/lib/python3.8/site-packages/mlflow/projects/__init__.py", line 92, in _run
submitted_run = backend.run(
File "/mnt/c/Users/username/Documents/venv/lib/python3.8/site-packages/mlflow/projects/backend/local.py", line 45, in run
work_dir = fetch_and_validate_project(project_uri, version, entry_point, params)
File "/mnt/c/Users/username/Documents/venv/lib/python3.8/site-packages/mlflow/projects/utils.py", line 126, in fetch_and_validate_project
project.get_entry_point(entry_point)._validate_parameters(parameters)
File "/mnt/c/Users/username/Documents/venv/lib/python3.8/site-packages/mlflow/projects/_project_spec.py", line 132, in get_entry_point
raise ExecutionException(
mlflow.exceptions.ExecutionException: Could not find main among entry points ['generate', 'train', 'complete'] or interpret main as a runnable script. Supported script file extensions: ['.py', '.sh']
2021/11/12 15:45:21 ERROR mlflow.cli: === Run (ID 'cd2b5b20f00344e5a6257deafabc4791') failed ===