Serverless express close mongodb connexion

user_1820957 0
Viewed 342

I am using serverless on aws with nodejs and mongodb atlas as database

At the moment I am using the trial version which allow maximum 500 connections.

Seems that my code is not disconnecting the database when process end

I am using express to manage it

First I had no connection close thinking that the connection will be closed automatically once the process end but no I had a lot of connections open.

Then I added a middleware to close my connections after the response has been sent, it was not working, I was thinking that serverless was stopping the process once the response was sent.

Not on each route I am closing mongo connection, for example

router.get('/website/:id/page', async (req, res, next) => {
  try {
    const pages = await pageDataProvider.findByWebsite(req.params.id);
    await mongodbDataProvider.close();
    res.json(pages);
  } catch (error) {
    next(error)
  }
})

This is how I handle connections with mongo

const MongoClient = require('mongodb').MongoClient
const config = require('../config')

const MONGODB_URI = config.stage === 'test' ?
  global.__MONGO_URI__ :
  `mongodb+srv://${config.mongodb.username}:${config.mongodb.password}@${config.mongodb.host}/admin?retryWrites=true&w=majority`;
const client = new MongoClient(MONGODB_URI);

let cachedDb = null;

module.exports.connect = async () => {

  if (cachedDb) return cachedDb;

  await client.connect();

  const dbName = config.stage === 'test' ? global.__MONGO_DB_NAME__ : config.stage;
  const db = client.db(dbName)
  cachedDb = db;

  return db;
}

module.exports.close = async () => {
  if (!cachedDb) return;

  await client.close();
  cachedDb = null;
}

I do not understand why I have so many connections open

serverless express node.js mongodb javascript
3 Answers

Step 1

Isolate the call to the MongoClient.connect() function into its own module so that the connections can be reused across functions. Let's create a file mongo-client.js for that:

mongo-client.js:

const { MongoClient } = require('mongodb');

// Export a module-scoped MongoClient promise. By doing this in a separate
// module, the client can be shared across functions.
const client = new MongoClient(process.env.MONGODB_URI);

module.exports = client.connect();

Step 2

Import the new module and use it in function handlers to connect to database.

some-file.js:

const clientPromise = require('./mongodb-client');

// Handler
module.exports.handler = async function(event, context) {
  // Get the MongoClient by calling await on the connection promise. Because
  // this is a promise, it will only resolve once.
  const client = await clientPromise;
  
  // Use the connection to return the name of the connected database for example.
  return client.db().databaseName;
}
user_143898300

I think its a programmatic error in your close method. Please have a closer look at

if (!cachedDb) return;

I think it should have been

if (cachedDb != null) return;

As stated in other response, I would strongly advice against closing the DB connections with each request. You should be looking for a pool mechanism, where a connection from the pool is handed to your application. The application can wait till it receives the connection

Closure of the DB connections should be handled at the time when the application is exiting (shutting/going down). This way application will at least try to close the connections gracefully.

Nonetheless, here is an adaptation your program

index.js

const express = require('express')
const app = express()
const port = 3000
const dbProvider = require('./dbProvider');

dbProvider.connect();

app.get('/testConnection',async (req, res, next) => {
    console.log('Doing something for fetching the request & closing connection');
    dbProvider.close();
    console.log('After closing the connection');
})

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})

dbProvider.js

let cachedDb = null;
let db = {};
module.exports.connect = async () => {
    if (cachedDb) {
        console.log('Returning Cachedb');
        return cachedDb;
    }
    else{
        console.log('Not a cachedDB');
    }
    db.setup = 1;
    return db;
}
module.exports.close = async () => {
    if (!cachedDb) {
        console.log('Since its cached DB not closing the connection');
        return;
    }
    db=null;
    return;
}

And here is the console output:

-> node index.js
Not a cachedDB
Example app listening at http://localhost:3000
Doing something for fetching the request & closing connection
Since its cached DB not closing the connection
After closing the connection
user_58144220
user_54174490
Related