I am writing unittest for a python class. I want to add database prefix because the original class has it. when i create a temporary view for the same. i am getting this error. Any pointers on how to specify the database prefix
Error
pyspark.sql.utils.AnalysisException: It is not allowed to add database prefix `union` for the TEMPORARY view name.
Code:
columns = ["company_id", "item_id", "ingest_date", "attachable_id"]
data = [
(100, 1, datetime(2021, 1, 1, 0, 0, 0, 0), 900),
(100, 2, datetime(2021, 1, 1, 0, 3, 0, 0), 903),
(100, 2, datetime(2021, 1, 1, 0, 1, 0, 0), 902),
(200, 1, datetime(2021, 1, 1, 0, 1, 0, 0), 900)
]
rowData = map(lambda x: Row(*x), data)
spark = SparkSession.builder.appName('SparkByExamples.com').getOrCreate()
df = spark.createDataFrame(rowData, columns)
df.createOrReplaceTempView("union.items_vw")
select_query = "SELECT count(*) from union.items_vw"
batch_df = spark.sql(select_query)
print(batch_df.show())
```