How do I count all the documents in a collection and use the cont in a controller, with MongoDB and Express.js?

Viewed 1285

I am working on a blogging application (click the link to see the GitHub repo) with Express (version 4.17.1), EJS and MongoDB (version 4.0.10).

Trying to paginate the posts I did the following, in the controller:

exports.getPosts = (req, res, next) => {

    const perPage = 5;

    const currPage = req.query.page ? parseInt(req.query.page) : 1;

    let postsCount = 0;

    const posts = Post.find({}, (err, posts) => {

            postsCount = posts.length;

            let pageDecrement = currPage > 1 ? 1 : 0;

            let pageIncrement = postsCount >= perPage ? 1 : 0;

            if (err) {
                console.log('Error: ', err);
            } else {
                res.render('default/index', {
                    moment: moment,
                    layout: 'default/layout',
                    website_name: 'MEAN Blog',
                    page_heading: 'XPress News',
                    page_subheading: 'A MEAN Stack Blogging Application',
                    currPage: currPage,
                    posts: posts,
                    pageDecrement: pageDecrement,
                    pageIncrement: pageIncrement
                });
            }
        })
        .sort({
            created_at: -1
        })
        .populate('category')
        .limit(perPage)
        .skip((currPage - 1) * perPage);
};

And in the view:

<a class="btn btn-primary <%= pageDecrement == 0 ? 'disabled' : '' %>" href="/?page=<%= currPage - pageDecrement %>">&larr; Newer Posts</a>

and

<a class="btn btn-primary <%= pageIncrement == 0 ? 'disabled' : '' %>" href="/?page=<%= currPage + pageIncrement %>">Older Posts &rarr;</a>

That works fine unless there are is a number of posts equal to perPage x N, where N is an integer, in which case the "Older Posts" button becomes disabled one page too late.

That is because postsCount = posts.length counts the posts after they are limited by .skip((currPage - 1) * perPage).

So I need to count the posts from the model/collection and bring that count variable in the controller.

My model:

const mongoose = require('mongoose');

const postSchema = new mongoose.Schema({
    title: {
        type: String,
        required: true
    },
    short_description: {
        type: String,
        required: true
    },
    full_text: {
        type: String,
        required: true
    },
    category: {
        type: mongoose.Schema.Types.ObjectId,
        ref: 'Category'
    },
    post_image: {
        type: String,
        required: false
    },
    updated_at: {
        type: Date,
        default: Date.now()
    },
    created_at: {
        type: Date,
        default: Date.now()
    }
});

module.exports = mongoose.model('Post', postSchema);

How do I count all the documents in the posts collection and use that number in the posts controller?

3 Answers

This can be done easier with mongodb aggregation framework.

We use $facet aggregation to get the paginated data along with the total number of documents.

In aggregation framework we use $lookup instead of mongoose populate. $lookup returns an array, to get the first item in array we use $arrayElemAt operator inside $addFields.

Playground

And here is the code to apply to your app: (The first $match aggregation is unnecessary here, but I put in in case you may need it in the future)

exports.getPosts = async (req, res, next) => {
    const perPage = 5;
    const currPage = req.query.page ? parseInt(req.query.page) : 1;
    const skip = (currPage - 1) * perPage;

    try {
        const result = await Post.aggregate([{
                $match: {},
            },
            {
                $sort: {
                    created_at: -1,
                },
            },
            {
                $lookup: {
                    from: "categories",
                    localField: "category",
                    foreignField: "_id",
                    as: "category",
                },
            },
            {
                $addFields: {
                    category: {
                        $arrayElemAt: ["$category", 0],
                    },
                },
            },
            {
                $facet: {
                    totalRecords: [{
                        $count: "total",
                    }, ],
                    data: [{
                            $skip: skip,
                        },
                        {
                            $limit: perPage,
                        },
                    ],
                },
            },
        ]);

        let postsCount = result[0].totalRecords[0].total;
        const pageCount = Math.ceil(postsCount / perPage);
        const pageDecrement = currPage > 1 ? 1 : 0;
        const pageIncrement = currPage < pageCount ? 1 : 0;
        const posts = result[0].data;

        res.render("default/index", {
            moment: moment,
            layout: "default/layout",
            website_name: "MEAN Blog",
            page_heading: "XPress News",
            page_subheading: "A MEAN Stack Blogging Application",
            currPage,
            posts,
            pageDecrement,
            pageIncrement,
        });
    } catch (err) {
        console.log("Error: ", err);
        res.status(500).send("something went wrong");
    }
};

By the way, in the post schema, for date fields you use default: Date.now(), this will cause the date value always the same value, it should be in this format: default: Date.now

Read $facet.

New in version 3.4.

Processes multiple aggregation pipelines within a single stage on the same set of input documents. Each sub-pipeline has its own field in the output document where its results are stored as an array of documents.

Example: See here

db.collection.aggregate([
  {
    $facet: {
      "count": [
        { $match: {} },
        { $count: "totalCount" }
      ],
      "data": [
        { $match: {} },
        { $sort: { _id: -1 } },
        { $skip: 1 },
        { $limit: 2 }
      ]
    }
  }
])

Mongoose Version:

Model.aggregate([
  {
    $facet: {
      "count": [
        { $match: {} },
        { $count: "totalCount" }
      ],
      "data": [
        { $match: {} },
        { $sort: { _id: -1 } },
        { $skip: 1 },
        { $limit: 2 }
      ]
    }
  }
]).
then(res => console.log(res)).
catch(error => console.error('error', error));

In case of Mongoose you should use this:

https://mongoosejs.com/docs/api.html#aggregate_Aggregate-facet

Official Mongodb docs:

https://docs.mongodb.com/manual/reference/operator/aggregation/facet

General idea is to perform aggregation instead of multiple calls (1 for getting needed info + 1 to get the total count of documents)

You can perform 2 separate calls of course but it will hit your performance (not much for small data volumes but still...) So you can get all needed data with .find() and then get count like this: https://mongoosejs.com/docs/api.html#model_Model.count

PS. btw, use async/await instead of callbacks to avoid callback hell

Related