How can I update application progress on Spark UI?

Viewed 157

In the Spark UI it looks like the application is always at about 10% in the progress bar.

Spark UI screenshot

enter image description here

To confirm real progress, I have to open the job on the UI, confirm the job/query in progress and estimate it from my experience of that specific job.

Spark job UI screenshot

enter image description here

I would like to be able to update that progress from the scala code to allow me to really know the actual progress.

0 Answers
Related