I'm using an private server to submit some jobs on databricks using databricks-connect. Where is host Spark driver ? On private server and it just sends jobs to databricks worker ? I'm able to see .show() results directly on my server, what does it mean ? Am I using client mode, and spark driver stay on private server ?
Thank you