i am using python code with psycopg2 to automate the process of downloading csv files from a website and copying to database but i need a way that does not allow duplicate data persistence, what would be the best way around this situation?
i am currently using this code to copy:
COPY table_name(item ,order ,cn_main,cn_second)
FROM '{dir.get_cn_folder()}'
DELIMITER ';'
CSV HEADER encoding 'ISO-8859-1';