How to trace submited Spark-SQL queries

Viewed 27

when people execute Spark-SQL by:

  1. ./bin/spark-sql to execute THE-QUERY on the command line OR
  2. ./bin/spark-submit -f X.sql to execute THE-QUERY by a sql file

What exactly THE-QUERY is cannot be seen on the Spark WebUI or history server.

It is hard to trace the source sql when exception happens. So I am wondering if Spark itself actually has some features to log the submitted SQL ?

0 Answers
Related