I'm working on importing a file into SQL Server from a s3 bucket. I have no problem picking up the file, but when I try to import it into SQL I keep getting the following error:
errorMessage": "(207, b"Invalid column name 'nan'.DB-Lib error message
My code looks like this:
client = boto3.client('s3')
my_bucket = 'address/staging'
filename = 'anjou_mailing_list.csv'
subscriber_csv = client.get_object(Bucket = my_bucket, Key = data_filename)
data_body = subscriber_csv["body"]
csv_string = data_body.read().decode('utf-8')
sub_df = pd.read_csv(StringIO(csv_string), sep = ',')
#Make db connection
conn = pymssql.connect(server = 'rds_sqlserver.com', user='username',password = 'my_password', database = 'mixology')
cursor = conn.cursor()
for index, row in sub_df.itterrows():
cursor.execute('''INSERT INTO dbo.anjou_class(firstname,lastname, email, street, city, state, zip, country_code)
VALUES(%s,%s,%s,%s,%s,%s,%s,%s)
'''
, (row['f_name']
, row['lname']
, row['email_address']
, row['street_ad']
, row['city']
, row['state']
, row['zip']
, row['country'])
)
cursor.commit()
Sample csv looks like such:
John,Doe,fakemail@gmail.com,Greenleaf CT,APT 204,Aurora,OH,48736,,
Beatrice,Winncons,heremail@gmail.com,2018 crafters,Shelbyville,IN,60505,,
I'm not getting a synatx error and when I run the code locally, my data is getting entered. But when I run it as a lambda function it's falling over. I thought it needed a quotechar for the apartment address so I added made a change sub_df = pd.read_csv(StringIO(csv_string), sep = ',', quotechar = '"') but that seems to fail too.
Thanks in advance