Throughput and rate limit for AWS ApiGatewayManagement.postToConnection (websockets)

user_2824661 0
Viewed 497

I've developed a site which hosts a two-player board game (backgammon) and while a match is being played other users can view the game and watch in real time. The AWS lambda that implements the game logic accepts moves from the players over websocket, stores in DynamoDB and uses ApiGatewayManagement.postToConnection to notify players and watchers about changes to the board state.

To send updates to watchers I am using Promise.all like this...

await Promise.all(watchers.map(to => gateway.postToConnection({
  ConnectionId: to,
  Data: <message>
})))

This is working fine for small numbers of watchers (10-20). My question is about how this will scale to 100s or even 1,000s of watchers. I can't absolutely confirm this, but looking at XRay it appears that using the method above, notification is done in batches of four concurrent postToConnection calls.

  • Does the node thread pool come into play in this scenario? I'm not 100% clear whether node will use NIO and fire off 100 backend requests for 100 notifications all at once, or put them on its internal thread pool which would limit concurrent requests to a small number. I am assuming here that postToConnection results in an http call to an AWS service and that call can use NIO.

  • Are calls to postToConnection rate limited? There is good information on inbound rate limits into API Gateway but I cannot find any information on outbound rate limits

Many thanks.

aws-api-gateway node.js amazon-web-services websocket
2 Answers
Admin1

Thanks Joe Nye for your answer, on re-reading the docs this does seem clear enough. To answer the first part we need to assume postToConnection makes an http request under the covers. I did a little test and it does appear that Promise.all will invoke any number of requests in parallel (subject to system resources).

To test it I wrote a little express server and client...

// server.js
const express = require("express")
const http = require("http")

const app = express()

const server = http.createServer(app)

function sleep(time) {
    return new Promise((resolve) =>{
        setTimeout(resolve, time)
    })
}

let conn=1
app.post("/", async (req, res) => {
    console.log("Serving", conn++)
    await sleep(5000)
    res.send("OK")
})

server.listen(9000, () => {
    console.log(`Server running on port: 9000`)
})

// client.js
const fetch = require("node-fetch")

async function run() {
    const promises = Array.from({length: 1000})
        .map((_, index) => fetch("http://localhost:9000", {method: "POST"})
            .then(r => console.log("Fetch", index, "complete")))
    
    return Promise.all(promises)
}

run().catch(err => console.log(err))

When I run the client, "Serving 0..999" appears immediately on the server console, then 5 seconds later "Fetch 0..999 complete" appears immediately on the client console.

user_28246610
Related