Kinesis triggered lambda runs below expected number of times

user_3492125 0
Viewed 36

I have a lambda configured to be triggered by a Kinesis stream at a batch size of 100 and batch window of 60s. The Kinesis stream has about 2 shards and 2K+ total records per minute. I'd imagine this would translate into about 20+ lambda runs. However, every minutes, I'm seeing only an average of 5-6 runs. enter image description here

I also see an iterationAge of 86K ms (approximately 24 Hours, my Kinesis stream holds records up to 24 hours) enter image description here

The lambda function is not a fast one, runs approximately 25s on average. enter image description here

I don't see any error count, or throttles or async delivery failures. enter image description hereenter image description here

Appreciate any advice on how to troubleshoot this.

amazon-kinesis aws-lambda amazon-web-services
1 Answers

I'd imagine this would translate into about 20+ lambda runs

It does not work like that. If you have 2 shards, you will have only two concurrent lambda functions (one shard = one function), otherwise your records would be processed out of order.

runs approximately 25s on average

That's why you have about 5-8 lambda invocations per minute:.

2 shards x 4 invocations for each function within 1 minute

You would have to increase parallelization factor for your function. But this requires your records to have different partition keys within a shard. If they have same partition key, parallelization will not work.

user_2488230
Related