Aggregation Query Optimization Mongodb

Viewed 66

User Schema

I have been building a social media application and I have to write a query that returns the user of user. The schema of user is shown below.

const userSchema = Schema(
  {
    email: {
      type: String,
      unique: true,
      required: [true, "Email is required"],
      index: true,
    },
    active: {
      type: Boolean,
      default: true,
    },
    phone: {
      type: String,
      unique: true,
      required: [true, "Phone is required"],
      index: true,
    },
    name: {
      required: true,
      type: String,
      required: [true, "Name is required"],
    },
    bio: {
      type: String,
    },
    is_admin: {
      type: Boolean,
      index: true,
      default: false,
    },
    is_merchant: {
      type: Boolean,
      index: true,
      default: false,
    },
    password: {
      type: String,
      required: [true, "Password is required"],
    },
    profile_picture: {
      type: String,
    },
    followers: [
      // meaning who has followed me
      {
        type: Types.ObjectId,
        ref: "user",
        required: false,
      },
    ],
    followings: [
      // meaning all of them who I followed
      {
        type: Types.ObjectId,
        ref: "user",
        required: false,
      },
    ],
  },
  {
    timestamps: { createdAt: "created_at", updatedAt: "updated_at" },
    toObject: {
      transform: function (doc, user) {
        delete user.password;
      },
    },
    toJSON: {
      transform: function (doc, user) {
        delete user.password;
      },
    },
  }
);

Follow/following implementation

I have implemented follow/following using the logic shown as below. Each time user follows another user. It would perform 2 queries. One would update the follower followers part using findOneAndUpdate({push:followee._id}) and a second query to update the part of followee user.

User Model after follow/following update

Query Response Pattern

I have written a query that should return the response with followings response appended to each user

{
  doesViewerFollowsUser: boolean // implying if person we are viewing profile of follows us 
  doesUserFollowsViewer: boolean // implying if person we are viewing profile of follows us
}

The actual query

The query must looks like this


userModel
    .aggregate([
      {
        $match: {
          _id: {
            $in: [new Types.ObjectId(userId), new Types.ObjectId(viewerId)],
          },
        },
      },

      {
        $addFields: {
          order: {
            $cond: [
              {
                $eq: ["$_id", new Types.ObjectId(viewerId)], // testing for viewer
              },
              2,
              1,
            ],
          },
        },
      },
      {
        $group: {
          _id: 0,
          subjectFollowings: {
            $first: "$followings",
          },
          viewerFollowings: {
            $last: "$followings",
          },
          viewerFollowers: {
            $last: "$followers",
          },
        },
      },
      {
        $lookup: {
          from: "users",
          localField: "subjectFollowings",
          foreignField: "_id",
          as: "subjectFollowings",
        },
      },
      {
        $project: {
          subjectFollowings: {
            $map: {
              input: "$subjectFollowings",
              as: "user",
              in: {
                $mergeObjects: [
                  "$$user",
                  {
                    doesViewerFollowsUser: {
                      $cond: [
                        {
                          $in: ["$$user._id", "$viewerFollowers"],
                        },
                        true,
                        false,
                      ],
                    },
                  },
                  {
                    doesUserFollowsViewer: {
                      $cond: [
                        {
                          $in: ["$$user._id", "$viewerFollowings"],
                        },
                        true,
                        false,
                      ],
                    },
                  },
                ],
              },
            },
          },
        },
      },
      {
        $project: {
          "subjectFollowings.followings": 0,
          "subjectFollowings.followers": 0,
          "subjectFollowings.bio": 0,
          "subjectFollowings.password": 0,
          "subjectFollowings.is_admin": 0,
          "subjectFollowings.is_merchant": 0,
          "subjectFollowings.email": 0,
          "subjectFollowings.phone": 0,
          "subjectFollowings.created_at": 0,
          "subjectFollowings.updated_at": 0,
          "subjectFollowings.__v": 0,
        },
      },
    ])

The problem

I don't think the current query scales that much. The worst case complexity for this query reaches 0(n^2) (approximately). So, please help me optimize this query.

1 Answers

The problem is with your data modeling. You shound not store follower/following in an array because:

  1. Mongodb has a 16mb hard limit for every document, which means you can store limited data in a single document
  2. Arrays lookups will take linear time; larger the array, longer it will take to query it.

What you can do is have a collection for user relationships like so:

follower: user id
followee: user id

You can then create a compound index on follower-followee and query effectively to check who follows who. You can also enable timestamps here. In order to get all followers of a user, just create an index on followee key and this will also resolve quickly

Related