I want to read all the Impala table name from Spark, using a JDBC connection.
This is an example of the code in Pyspark:
df = spark.read.format('jdbc').option('url','jdbc:impala://172.25.0.1:21050/default').option('driver','com.cloudera.impala.jdbc41.Driver').option('dbtable', '(show tables) t').load()
And this is the error that appears:
Caused by: com.cloudera.impala.support.exceptions.GeneralException: [Cloudera][ImpalaJDBCDriver](500051) ERROR processing query/statement. Error Code: 0, SQL state: TStatus(statusCode:ERROR_STATUS, sqlState:HY000, errorMessage:ParseException: Syntax error in line 1:
SELECT * FROM (show tables) t WHERE 1=0
^
Encountered: SHOW
Expected: SELECT, VALUES, WITH
CAUSED BY: Exception: Syntax error
), Query: SELECT * FROM (show tables) t WHERE 1=0.
... 32 more
It seems that the "SHOW" command is not supported. Is there another way to achieve the goal?