Mongoose: Why isn't populate() working for my self-referencing model?

user_355096 0
Viewed 122

As far as I can tell, populate() is being called in my code (because I get an error if I give it a wrong path), but it doesn't seem to be doing anything.

I searched for past question in Stack Overflow, and I've not seen one where someone's using a model that's referencing itself, so my guess is that that might be the problem.

This Mongoose doc is where I'm reading up on how to use populate().

My Model

const mongoose = require('mongoose');

const schema = new mongoose.Schema({
    firstName: { type: String },
    lastName: { type: String },
    email: { type: String, unique: true },
    teamLeaders: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Agent' }],
    teamMembers: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Agent' }]
});

let Agent = mongoose.model('Agent', schema);
Agent.init();

module.exports = Agent;

The actual document in MongoDB Atlas (anonymised name + email)

{
  "_id": {
    "$oid": "62e3e0ab57560a5c15a535e0"
  },
  "teamLeaders": [],
  "teamMembers": [
    {
      "$oid": "62e3f548678dbed5593acc8e"
    },
    {
      "$oid": "62e3f548678dbed5593acc91"
    },
    {
      "$oid": "62e3f548678dbed5593acc94"
    },
    {
      "$oid": "62e3f548678dbed5593acc97"
    },
    {
      "$oid": "62e3f548678dbed5593acc9a"
    },
    {
      "$oid": "62e3f548678dbed5593acc9d"
    },
    {
      "$oid": "62e3f548678dbed5593acca0"
    },
    {
      "$oid": "62e3f548678dbed5593acca3"
    }
  ],
  "firstName": "John",
  "lastName": "Smith",
  "email": "john@smith.com",
  "__v": 8
}

Code where I'm calling populate()

const Agent = require('../models/agents');

const mongoose = require("mongoose");
const db = require("../config/db");
mongoose.connect(process.env.MONGODB_URI || db.url);

// I've removed other functions that are not related to this. And the DB connection is definitely working fine.

// Actual private function in my code.
async function addAgent(firstName, lastName, email, isTeamLeader, teamLeader) {
    let newAgent = Agent();

    newAgent.firstName = firstName;
    newAgent.lastName = lastName;
    newAgent.email = email;

    if (isTeamLeader) {
        await newAgent.save();
    } else {
        newAgent.teamLeaders.push(teamLeader);

        let savedAgent = await newAgent.save();

        teamLeader.teamMembers.push(savedAgent);
        await teamLeader.save();
    }
}

// This is a dummy function to show how I created the agents.
async function createAgents() {
    await addAgent('John', 'Smith', 'john@smith.com', true, null);

    // Some time later... I called addAgent() manually since this is for an internal team with only 30 people.
    // It's also why I'm just querying for the firstName since there's only one John in the internal team.
    let teamLeader = await Agent.findOne({ firstName: 'John' });
    await addAgent('Peter', 'Parker', 'peter@parker.com', false, teamLeader);
}

// This is the main one where I try to call populate().
async function mainFunction() {
    Agent.findOne({ firstName: 'John' }).populate({ path: 'teamMembers', model: 'Agent' }).exec((err, agent) => {
        if (err) return handleError(err);
        console.log('Populated agent: ' + agent);
    });
}
mongoose node.js mongodb javascript
2 Answers

The data you got does bot correspond to the data you have. How dhould mongoose know that the reference id is inside oid?

With

teamLeaders: [{ type: mongoose.Schema.Types.ObjectId, ref: 'Agent' }],

one should expect inside atlas something like

"teamLeaders": [
    ObjectId("62e3f548678dbed5593acca0"),
    ObjectId("62e3f548678dbed5593acca3"),
    ...
  ],

Make sure the ref is exactly referencing the correct value.

Anyway I'd use an aggregation, as it is probably way more efficientand can be used also with mongoose if you have to

https://www.mongodb.com/docs/manual/aggregation/

user_109093170

Thanks to questions/prompt from @Weedoze, I've figured out where the issues lie.

There are two main problems.

Problem 1

I had completely misunderstood the Mongoose docs for populate().

My initial understanding was that calling populate() will update the actual MongoDB document with the populated result. However, what it does is that it's a quality-of-life/convenience feature where it does a second query to replace the reference with the actual document contents. All of these only exist/happen at runtime.

Problem 2

populate() is doing its thing here, the problem is that the callback in exec() isn't actually running - I'm still pretty stumped by this as I can't see a reason why it wouldn't work, and I'm doing the same thing as the doc (as well as other guides on the internet).

To clarify, this means that the populate() function does run, but the callback function inside exec() isn't being reached at all (I've checked by blocking it with a debugger checkpoint).

Ways I'm using the callback which do not work.

Agent.findOne({ firstName: 'John' }).populate('teamMembers').exec((err, agent) => {
    if (err) return handleError(err);
    console.log('Populated agent: ' + agent);
});

Agent.findOne({ firstName: 'John' }).populate('teamMembers').exec(function(err, agent) {
    if (err) return handleError(err);
    console.log('Populated agent: ' + agent);
});

To get around this, I just access the results in other ways, e.g.

let agent = await Agent.findOne({ firstName: 'John' }).populate('teamMembers');
console.log('Populated agent: ' + agent);

// Pretty much same thing as above. Only difference is that exec() returns a proper Promise.
// Tried this mostly to confirm that exec() does run and the issue is with the callback.
let agent = await Agent.findOne({ firstName: 'John' }).populate('teamMembers').exec();
console.log('Populated agent: ' + agent);

// Since populate() returns a "then-able" result anyway.
Agent.findOne({ firstName: 'John' }).populate('teamMembers').then((err, agent) => {
    if (err) return handleError(err);
    console.log('Populated agent: ' + agent);
});
user_3550960
Related