I am running an ML pipeline, at the end of which I am logging certain information using mlflow. I was mostly going through Databricks' official mlflow tracking tutorial.
import mlflow
import mlflow.sklearn
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error
with mlflow.start_run():
n_estimators = 100
max_depth = 6
max_features = 3
# Create and train model
rf = RandomForestRegressor(n_estimators = n_estimators, max_depth = max_depth, max_features = max_features)
rf.fit(X_train, y_train)
# Make predictions
predictions = rf.predict(X_test)
# Log parameters
mlflow.log_param("num_trees", n_estimators)
mlflow.log_param("maxdepth", max_depth)
mlflow.log_param("max_feat", max_features)
# Log model
mlflow.sklearn.log_model(rf, "random-forest-model")
# Create metrics
mse = mean_squared_error(y_test, predictions)
# Log metrics
mlflow.log_metric("mse", mse)
When I run the above block of code in Databricks notebook, the below status message shows:
(1) MLflow run
Logged 1 run to an experiment in MLflow. Learn more
And I can view the logged information by clicking on "1 run."
However, I would like to automatically retrieve this link. In particular, I need the link to the mlflow uri where the artifacts are stored. This link is in the following format:
https://mycompany-dev.cloud.databricks.com/?o=<ID_1>#mlflow/experiments/<ID_2>/runs/<ID_3>
I tried investigating the url and finding the various id codes that are present in it by printing the following information:
print("Tracking URI: ", mlflow.get_tracking_uri())
print("Run id:", run.info.run_id)
print("Experiment:", run.info.experiment_id)
I figured out that <ID_2> in the link above is the experiment_id and <ID_3> is the run_id. But I have no idea what <ID_1> stands for. Also, I believe there should be a built-in functionality to retrieve the link of saved artifacts, instead of manually having to build up the link from sections. However, I haven't found such a funcitonality in the documentation so far.
Edit: Now I discovered that <ID_1> is the Databricks workplace id. But it is still a question how I can access it programatically.