Lambda failing to import CSV to RDS SQL Server

Viewed 130

I'm working on importing a file into SQL Server from a s3 bucket. I have no problem picking up the file, but when I try to import it into SQL I keep getting the following error:

errorMessage": "(207, b"Invalid column name 'nan'.DB-Lib error message

My code looks like this:

client = boto3.client('s3')
my_bucket = 'address/staging'
filename = 'anjou_mailing_list.csv'
subscriber_csv = client.get_object(Bucket = my_bucket, Key = data_filename)
data_body = subscriber_csv["body"]
csv_string = data_body.read().decode('utf-8')
sub_df = pd.read_csv(StringIO(csv_string), sep = ',')
#Make db connection
conn = pymssql.connect(server = 'rds_sqlserver.com', user='username',password = 'my_password', database = 'mixology')
cursor = conn.cursor()
  for index, row in sub_df.itterrows():
     cursor.execute('''INSERT INTO dbo.anjou_class(firstname,lastname, email, street, city, state, zip, country_code)
                        VALUES(%s,%s,%s,%s,%s,%s,%s,%s)
                        '''
                        , (row['f_name']
                        , row['lname']
                        , row['email_address']
                        , row['street_ad']
                        , row['city']
                        , row['state']
                        , row['zip']
                        , row['country'])
                        )
cursor.commit()

Sample csv looks like such:

John,Doe,fakemail@gmail.com,Greenleaf CT,APT 204,Aurora,OH,48736,,
Beatrice,Winncons,heremail@gmail.com,2018 crafters,Shelbyville,IN,60505,,

I'm not getting a synatx error and when I run the code locally, my data is getting entered. But when I run it as a lambda function it's falling over. I thought it needed a quotechar for the apartment address so I added made a change sub_df = pd.read_csv(StringIO(csv_string), sep = ',', quotechar = '"') but that seems to fail too.

Thanks in advance

0 Answers
Related