None-unique column index error using cur.fetch_pandas_all() to extract data from Snowflake

user_12906895 0
Viewed 727

I'm pulling results from Snowflake using the code below. The SQL statement returns fine in Snowflake Web UI but when pulled in Python I get a ValueError: Found non-unique column index

If this happened in my Pandas workflow, I would reset_index() but, as its happening at the point of the dataframe being created I'm unsure how to fix this.

As an interim, I'm exporting a csv from Snowflake and importing into Python which I hate doing as I'm refreshing multiple times a week.

cur.execute(sql)
df = cur.fetch_pandas_all()
pandas sql python
2 Answers

I discovered, that although snowflake allows the return of identical named columns in the webui, this is not compatible with pandas.

Just renaming in the SQL statement as below resolved the issue.

Select XXX as XXX_2

user_129068950

Have just tried forming the df like this and it worked.

cur.execute(sql)
df = pd.DataFrame.from_records(iter(cur), columns=[x[0] for x in cur.description])
user_169772310
Related