I have to read a table every 30 minutes and I have made the query which only takes updates rows.
dfsql = client.query(sql, project=project_id).to_dataframe()
I have already existing table when I created dataframe for the first time using this
dfsql.to_csv('/home/analytics/divyang/divyangtest.csv')
Now,I want to write the updated records into a new table (not in divyangtest.csv) so each time my query runs, I should have a new table like divyangtest1,divyangtest2 etc
You could try adding the time to it - guaranteeing a unique name and making it easier to understand when the file was made.
Make the filename something like this:
filename = f"divyangtest-{datetime.datetime.now()}.csv"
I just ran it and it came up with this:
'divyangtest-2020-06-17 13:14:37.114370.csv'
You can tidy it up to make the file to be whatever you like as well.
Here's a (messy) example that makes it look neat:
filename = f"divyangtest_{str(datetime.datetime.now()).split('.')[0].replace(' ', '_')}.csv"
This makes the filename look like this:
'divyangtest_2020-06-17_13:17:25.csv'
Let me know if you have any more questions in the comments.
Try this:
filename = f"divyangtest_{str(datetime.datetime.now()).split('.')[0].replace(' ', '_')}.csv"
dfsql.to_csv(filename)