describe spark table/view comment

user_12894926 0
Viewed 131

We can create a table and view it with a comment describing it. For example (from spark docs):

CREATE TABLE student (id INT, name STRING, age INT) USING CSV
    COMMENT 'this is a comment'
    TBLPROPERTIES ('foo'='bar');

How can you retrieve the comment in a "clean format"? By clean format, I mean only (or almost only) the table name and the comment describing it. Any other solution I've found bloats me with all the column types and information (which I don't need in this case).

I've tried:

DESCRIBE student
DESCRIBE EXTENDED student
SHOW CREATE TABLE student
DESCRIBE DETAIL student -- databricks only
SHOW VIEWS FROM default -- to try seeing views description
SHOW TABLES FROM default -- to try seeing tables description

The best would be to have something like SHOW TABLE/SHOW VIEWS, but with a column adding the description.

Is there an out-of-the-box solution for this? If not, is there a good custom way to achieve it?

Thank you.

databricks apache-spark-sql apache-spark
1 Answers

There is no way to get table comment only. However it's fairly easy to filter it out of DESCRIBE TABLE statement using Scala.

spark.sql("CREATE TABLE student (id INT, name STRING, age INT) USING CSV COMMENT 'this is a comment'")
spark.sql("DESCRIBE TABLE EXTENDED student").filter($"col_name" === "Comment").show

+--------+-----------------+-------+
|col_name|        data_type|comment|
+--------+-----------------+-------+
| Comment|this is a comment|       |
+--------+-----------------+-------+
user_28900930
Related