Trigger a Java REST endpoint upon S3 File Upload Event

Question

I have a Spring Batch application which does followings,

A different application puts a csv file in a S3 bucket INBOUND
Spring Batch application reads a csv file
using data of each row, it calls an API endpoint and get a quote
Put those quotes in a newly created csv and put in a S3 bucket OUTBOUND

Spring Batch Application Tech Stack

Created using Spring Boot
Spring Batch Job Launcher is called upon a REST endpoint
Deployed in OpenShift

What I want to do,

When the other application puts a new file into the S3 INBOUND bucket it needs to trigger the Spring Batch Operation (call the REST endpoint which launches the job)

Can I do this using a K8s Cron Job which triggered by S3 events? If so can someone please explain how? Or do I have any other options?

user_5019386 · Answer

I am not too familiar with AWS services but I know that they provide a notification service for S3 buckets: https://docs.aws.amazon.com/AmazonS3/latest/user-guide/enable-event-notifications.html

One of the event notification destinations is AWS Lambda. So you can try to create a function that calls the rest endpoint and which is triggered when a notification of type ObjectCreated arrives.

user_934005 · Answer

My solution was to create a lambda function which is triggered when a .txt file is uploaded into the S3 bucket's INBOUND folder.

Wrote a python handler to curl command to hit the job launcher REST end point.

Trigger a Java REST endpoint upon S3 File Upload Event

2 Answers