Select a column starting with '@' in databricks sql

user_7172424 0
Viewed 186

I'm getting to my through logstash into elasticsearch. I import the data in databricks, first through a python scan and then transforming the data into a dataframe The dataframe is then converted into temporary view within databricks. This view can then be used in sql modeling. This process can't be altered

When I query: Select * from tempview then I get all columns :)

enter image description here

Now the challenge: One of my field names is called @timestamp and I can see the databricks display that the column is there, but I can not select the column itself. In T-SQL I would have been able to use squarebrackets [@timestamp] and move on, but it does not work in hivesql.

I know that the @name syntax is reserved for sql variables, but is there a workaround?

logstash hiveql elasticsearch databricks
1 Answers

Use backticks `

select `@timestamp`, `@version` from t

Spark shell example:

Seq((1, "a", "10")).toDF("id", "@version", "@timestamp").registerTempTable("t")
sql ("select `@timestamp`, `@version` from t").show

+----------+--------+
|@timestamp|@version|
+----------+--------+
|        10|       a|
+----------+--------+
user_45583580
Related