Where can you change the batch size for an SQS queue that triggers an AWS Lambda function?

user_1580587 0
Viewed 8822

I could have sworn that there was an easy way to change the batch size of an SQS queue that is configured as a Lambda trigger, but as of July 2020 I can no longer find where this happens. This may have something to do with the "new SQS console experience" that is currently being advertised on the top of AWS.

I am able to see the current batch size for my queue (screenshot), but the number is not editable. I don't see anything related to batch size in the SQS interface either. It's possible that my current IAM credential does not have the ability to change batch sizes and it's hidden from me. Does anyone know where this value can be changed?

Screenshot from AWS showing where the current batch size is displayed

amazon-sqs aws-lambda amazon-web-services
3 Answers

This limitation would not be related to Amazon SQS console.

It would be related to AWS Lambda, since the Lambda service is responsible for polling the SQS queue and for specifying the batch size to retrieve.

You are correct that there is no function to edit the batch size in the Lambda console.

From update-event-source-mapping — AWS CLI Command Reference, here is an AWS CLI command that can update the batch size:

aws lambda update-event-source-mapping \
    --uuid  "a1b2c3d4-5678-90ab-cdef-11111EXAMPLE" \
    --batch-size 8

Output:

{
    "UUID": "a1b2c3d4-5678-90ab-cdef-11111EXAMPLE",
    "StateTransitionReason": "USER_INITIATED",
    "LastModified": 1569284520.333,
    "BatchSize": 8,
    "State": "Updating",
    "FunctionArn": "arn:aws:lambda:us-west-2:123456789012:function:my-function",
    "EventSourceArn": "arn:aws:sqs:us-west-2:123456789012:mySQSqueue"
}

Or, just delete the trigger in the Lambda console and create a new one.

user_1747770

SQS is not an exception. The same goes for DynamoDB and Kinesis streams. I think the reason is that all three services work with lambda through event source mappings. Nothing else is using the mappings.

However, updating event source mappings through console is also not possible. You can use CLI or SDK for that, but this also requires getting UUID of the mapping to modify. Sadly UUID is not provided in console either.

To use in CLI, you have to do it in two steps.

1 Get UUID

aws lambda list-event-source-mappings --query 'EventSourceMappings[].[UUID, EventSourceArn]' --output table

-------------------------------------------------------------------------------------------------------------------------------
|                                                   ListEventSourceMappings                                                   |
+---------------------------------------+-------------------------------------------------------------------------------------+
|  5ab44863-82c2-4acc-b9dc-b14ad368effa |  arn:aws:kinesis:us-east-1:xxxxx:stream/kstream                              |
|  7479947c-bde5-4041-a438-5eb08f350505 |  arn:aws:dynamodb:us-east-1:xxxx:table/test/stream/2020-07-28T23:13:41.006  |
|  40040139-32fb-4297-b094-3f08368c980c |  arn:aws:sqs:us-east-1:xxxxx:Messages                                        |
|  a2b22aa6-f37a-4603-895b-3a044661ebdf |  arn:aws:sqs:us-east-1:xxx:test-queue                                      |
+---------------------------------------+-------------------------------------------------------------------------------------+

2. Update the mapping (e.g. for SQS)

aws lambda update-event-source-mapping --uuid a2b22aa6-f37a-4603-895b-3a044661ebdf --batch-size 5

{
    "UUID": "a2b22aa6-f37a-4603-895b-3a044661ebdf",
    "BatchSize": 5,
    "EventSourceArn": "arn:aws:sqs:us-east-1:xxx:test-queue",
    "FunctionArn": "arn:aws:lambda:us-east-1:xxxx:function:testsfd",
    "LastModified": 1595978738.458,
    "State": "Updating",
    "StateTransitionReason": "USER_INITIATED"
}
user_2488230

Not sure about before but the lambda UI now does allow you configure the batch size.

Caveat: you have to manually create the trigger from the lamdba UI and not from the SQS UI.

Steps as outlined in the document

  1. Open the Functions page on the Lambda console.
  2. Choose a function.
  3. Under Designer, choose Add trigger.
  4. Choose a trigger type.
  5. Configure the required options and then choose Add.

Configuring a queue as an event source

user_11774410
Related