Null values in CSV causing issue in moving data to mysql through python

Viewed 30

I don't have control over the null values in csv because it being uploaded by different system, but when I try to move the data from csv to mysql facing issue in null values. It is a nullable column there is no constraint. Sample values:

subject_name subject_code subject_type description Pyro PY Major Pyrotechnic Techy Elective This is a subject Biology BIO Major This is a subject Zoology Zoo Major This is a subject

Python insert's first row and terminating with mysql error message.

Error Message: There is no traceback I just received below error message
Error while connecting to MySQL 1054 (42S22): Unknown column 'nan' in 'field list'

Note: code is working fine if there is no null in csv. sample data

import pandas as pd
import sys
import datetime
filePath = str(sys.argv[1])
df = pd.read_csv(filePath)
df.head()

for i,row in df.iterrows():
        #here %S means string values
        value = None
        sql = "INSERT INTO simple.subj_stg VALUES (%s,%s,%s,%s,%s,%s,%s,%s)"
        cursor.execute(sql, tuple(row.to_list() + [cre_usr_id,actn_usr_id,updt_usr_id,cre_ts]) )
        print("Record inserted")
        # the connection is not auto committed by default, so we must commit to save our changes
        conn.commit()
1 Answers

full the empty columns with df.fillna("Null", inplace = True) then run the insert statement to mysql from python

Related