has anyone ran into this error while reading a delta lake table using spark.sql??
I am running spark on an EMR and using S3 to store my tables. I am able to create the DB and tables to read from my S3 location. However, it's when i try to query the table, i constantly run into this error. When I create non delta tables, I am able to query using spark.sql just fine. This error is specific to querying delta tables.
ERROR -----
An error was encountered:
Table does not support reads: poc.contact
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/dataframe.py", line 485, in show
print(self._jdf.showString(n, 20, vertical))
File "/usr/lib/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py", line 1305, in __call__
answer, self.gateway_client, self.target_id, self.name)
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py", line 117, in deco
raise converted from None
pyspark.sql.utils.AnalysisException: Table does not support reads: poc.contact