How can I use AWS Lambda to read a csv in S3 and output the record count to HTML?

user_10791600 0
Viewed 657

I'm very new to AWS and this question could very well be impossible in AWS. I'm looking to create a static webpage with S3 that will show how many records are in a csv file. The csv is some fake data about employee attrition, so every row corresponds to an employee. The csv is already in an S3 bucket, and I have already set up the bucket to show a simple static webpage (see below).

Is it possible to use Lambda or anything else to display this record count on the webpage? Seems simple enough, yet I am struggling.

Below is my current webpage, and ideally I would like the record count to be where the big red "X" is.

Ideal Output

Here is my lambda function so far:

import json
import os
import boto3
import csv

def lambda_handler(event,  context):
    for record in event['Records']:
        bucket = record['s3']['bucket']['fakeemployee']
        file_key = record['s3']['object']['Employee Attrition Data.xlsx']
        s3 = boto3.client('s3')
        csvfile = s3.get_object(Bucket=bucket, Key=file_key)

Here is my test configuration in AWS so far:

{
  "Records": [
    {
      "s3": {
        "bucket": {
          "name": "fakeemployee",
          "arn": "arn:aws:s3:::fakeemployee"
        },
        "object": {
          "key": "Employee Attrition Data.xlsx",
          "size": 242.1,
          "eTag": "3df14b4d8bda007b946b9f176b89c9b5",
          "sequencer": "0A1B2C3D4E5F678901"
        }
      }
    }
  ]
}

Any advice?

EDIT: I have tried to create a API HTTP Gateway but have run into a "message not found" error when invoking. I used this tutorial to create the API gateway.

aws-lambda amazon-web-services amazon-s3
1 Answers

From what I can understand, the use-case is that when ever a new CSV file (I assume the file is few megabytes, not in gigabytes) is uploaded to your bucket, you would like to invoke a lambda function which is going to read the file and show how many records are there. The number of records should be used to update a static HTML file in S3.

The steps to achieve that are:

  1. Setup S3 Event Notifications with s3:ObjectCreated:* to trigger your lambda function whenever a CSV file uploaded.
  2. The event will trigger your lambda, and the function should use get_object boto3 call to get the CSV file. Check the number of records in the file by reading the CSV file.
  3. In the same lambda, read in your HTML static file from S3 using get_object. Update the file in lambda, and then save the new HTML file modified into your bucket using put_object.

For that, the lambda needs permissions to read and write to S3, as well as permissions for being invokable by S3 Events.

Other solutions are possible (e.g. with apigateway, dynamodb, etc), but may not be necessary at this stage of your development.

user_2488230
Related