aws python lambda: reading csv file (iterator should return strings)

Viewed 427

I'm getting this message when I'm trying to test my python 3.8 lambda function:

Logs are:

soc-connect
contacts.csv
{'ResponseMetadata': {'RequestId': '9D7D7F0C5CB79984', 'HostId': 'wOd6HvIm+BpLOMKF2beRvqLiW0NQt5mK/kzjCjYxQ2kHQZY0MRCtGs3l/rqo4o0r4xAPuV1QpGM=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'wOd6HvIm+BpLOMKF2beRvqLiW0NQt5mK/kzjCjYxQ2kHQZY0MRCtGs3l/rqo4o0r4xAPuV1QpGM=', 'x-amz-request-id': '9D7D7F0C5CB79984', 'date': 'Thu, 26 Mar 2020 11:21:35 GMT', 'last-modified': 'Tue, 24 Mar 2020 16:07:30 GMT', 'etag': '"8a3785e750475af3ca25fa7eab159dab"', 'accept-ranges': 'bytes', 'content-type': 'text/csv', 'content-length': '52522', 'server': 'AmazonS3'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2020, 3, 24, 16, 7, 30, tzinfo=tzutc()), 'ContentLength': 52522, 'ETag': '"8a3785e750475af3ca25fa7eab159dab"', 'ContentType': 'text/csv', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x7f858dc1e6d0>}
1153
<_csv.reader object at 0x7f858ea76970>
[ERROR] Error: iterator should return strings, not bytes (did you open the file in text mode?)

The code snippet is:

import boto3
import csv

def digest_csv(bucket_name, key_name):
    # Let's use Amazon S3
    s3 = boto3.client('s3');

    print(bucket_name)
    print(key_name)

    s3_object = s3.get_object(Bucket=bucket_name, Key=key_name)
    print(s3_object)

    # read the contents of the file and split it into a list of lines
    lines = s3_object['Body'].read().splitlines(True)
    print(len(lines))

    contacts = csv.reader(lines, delimiter=';')
    print(contacts)

    # now iterate over those contacts
    for contact in contacts:
        # here you get a sequence of dicts
        # do whatever you want with each line here
       print('-*-'.join(contact))

I think the problem is on csv.reader.

I'm setting first parameter an array of lines... Should it be modified?

Any ideas?

1 Answers

Instead of using the csv.reader the following worked for me (adjusted for your delimiter and variables):

for line in lines:
    contact = ''.join(line.decode().split(';'))
    print(contact)
Related