I have followed this tutorial to create my first scheduled Vertex AI Pipeline to run every minute. The only thing it does is prints "Hello, <any-greet-string>" and also returns this same string. I can see that it is running because the last run time updates and the last run result is "Success" every time.
My question is very simple: Where can I see this string printed and the output of my pipeline?
The output of the print() statements in the pipeline can be found in "Cloud Logging" with the appropriate filters. To check logs for each component in the pipeline, click on the respective component in the console and click "VIEW LOGS" in the right pane. A new pane with the logs will open in the pipeline page which will allow us to see the output of the component. Refer to the below screenshot.
I ran a sample pipeline from this codelab, Intro to Vertex Pipelines and below is the output for one of the print() statements in the pipeline.
Every component in a pipeline run is deployed as an individual Vertex AI custom job. Corresponding to the sample pipeline consisting 3 components, there are 3 entries in the "CUSTOM JOBS" section as shown below.
Therefore, to view the logs on the run level, we would need to query the log entries with the respective job_ids of the pipeline components and the job_id of the Cloud Scheduler job. The query would look like this.
resource.labels.job_id=("JOB_ID_1" OR "JOB_ID_2" [OR "JOB_ID_N"...])
severity>=DEFAULT
If there are no simultaneous pipeline runs, a simpler query like below can be used.
resource.type=("cloud_scheduler_job" OR "ml_job")
severity>=DEFAULT