Intermittent (time dependent) error when using Node Package rate-limiter-flexible in Express Node.js application

Viewed 61

My overall goal is to apply the basic logic of the package "rate-limiter-flexible" for my Express Node.js application. I've gotten the basic functionality with the "Memory" version which does not require any communication with a database (IP address counts stored in Node server memory). Now I'm trying to get the functionality to work with the MongoDB rate limiter modules (using an instance of rateLimiterMongo object from the package).

The problem I'm encountering is that my rate limiter middleware is throwing an error intermittently... The only pattern I can find is that the error occurs more frequently if there are > ~10 seconds between requests to the Node app. This is the error which occurs every 5-10 requests:

[RateLimiter Error]: Rejection object:

TypeError: Cannot read properties of null (reading 'points')

at RateLimiterMongo._getRateLimiterRes (C:\root\node_modules\rate-limiter-flexible\lib\RateLimiterMongo.js:124:33)

at RateLimiterMongo._afterConsume (C:\root\node_modules\rate-limiter-flexible\lib\RateLimiterStoreAbstract.js:51:22)

at C:\root\node_modules\rate-limiter-flexible\lib\RateLimiterStoreAbstract.js:205:16

at processTicksAndRejections (node:internal/process/task_queues:96:5)

So far, I have tried:

  1. Disabling buffering with Mongoose (was a recommendation from the package docs) -- did not work

  2. Changing from MongoDB Atlas free tier to a locally hosted MongoDB instance -- this resolved all occurrences of the error, but I need to be able to use the cloud service

Here is a minimal reproduction of the error I'm facing when connecting to a MongoDB Atlas free tier cluster (via MONGO_DB_URL):

// Node packages:
require('dotenv').config();
const { RateLimiterMongo } = require('rate-limiter-flexible');
const mongoose = require('mongoose');
const express = require('express');
const app = express();

// open a Mongoose connection and save it:
const dbUrl = process.env.MONGO_DB_URL;
const connectDB = async function () {
    await mongoose
        .connect(dbUrl, {
            // options
        })
        .catch(error => {
            console.log("DB not connected!");
            // handle error here (initial connection)
        });
};
connectDB();
const mongoConn = mongoose.connection;

// options and passing to the RateLimiterMongo constructor:
const opts = {
    storeClient: mongoConn,
    points: 3, // Number of points
    duration: 1, // Per second(s)
};
const rateLimiterMongo = new RateLimiterMongo(opts);

// create the middleware for the express app:
const rateLimiterMiddleware = (req, res, next) => {
    rateLimiterMongo.consume(req.ip, 1)
        .then((rateLimiterRes) => {
            console.log("[RateLimiter Success]: RateLimiterRes object:\n", rateLimiterRes);
            next();
            // Allowed
        })
        .catch((rej) => {
            console.log("[RateLimiter Error]: Rejection object:\n", rej);
            res.status(500).send("RateLimiter error(s)...");
            // Blocked
        });
};

// Express app code:
app.use(rateLimiterMiddleware);
app.get('/', (req, res) => {
    res.status(200).send("Valid Route!");
});
app.listen(3000, () => {
    console.log(`Serving on port 3000!`);
});

Thanks all for any help you can provide with this. It may just be a side effect of using the MongoDBAtlas free tier...

1 Answers

Most likely, you use mongoose v6.x, which changes how connection established. It returns promise, which should be awaited, before connection can be used to make queries. More context in migration to 6 guide.

You should await connectDB() call and only then create middleware. In other words, your Express app should wait until connection to MongoDB Atlas is established. You can read through comments in related closed issue on github.

Related