$lookup returns an empty array mongoose

Question

I am using the following code with $lookup function.

        postSchemaModel.aggregate([{
            "$geoNear": {
                "near": { "type": "Point", "coordinates": [6.7336665, 79.8994071], "Typology": "post" },
                "distanceField": "dist.calculated",
                "maxDistance": 5000,
                "includeLocs": "dist.location",
                "spherical": true
            }
        },
        { "$limit": limit },
        { "$skip": startIndex },
        { "$sort": { "createdAt": -1 } },
        {
            "$lookup": {
                "from": userSchemaModel.collection.name,
                "localField": "user_id",
                "foreignField": "_id",
                "as": "user_id"
            }
        },
        {
            "$project": {
                "post_data": 1,
                "likes": 1,
                "commentsCount": 1,
                "post_img": 1,
                "isUserLiked": 1,
                "usersLiked": 1,
                'exp_date': 1,
                "has_img": 1,
                "user_id": "$user_id",
                "typology": 1,
                "geometry": 1,
                "category": 1,
                "created": 1,
                "createdAt": 1,
                "updatedAt": 1,
            }
        },
    ]).then(async function(posts) {
        //some code here
    });

The problem is this gives me an empty array for user_id. The following is one output I receive.

{ _id: 5ee1f89732fd2c33bccfec55,
    post_data: 'vvhhh',
    likes: 1,
    commentsCount: 0,
    post_img: null,
    isUserLiked: false,
    usersLiked: [ 5edf43b93859680cf815e577 ],
    exp_date: 2020-06-12T18:30:00.000Z,
    has_img: false,
    typology: 'chat',
    geometry:
     { pintype: 'Point',
       _id: 5ee1f89732fd2c33bccfec56,
       coordinates: [Array] },
    category: [],
    created: 1591867543468,
    createdAt: 2020-06-11T09:25:43.478Z,
    updatedAt: 2020-06-15T10:01:01.133Z,
    user_id: [] }

In my case I don't want it to be null and I am expecting a output like below.

{ _id: 5ee1f89732fd2c33bccfec55,
post_data: 'vvhhh',
likes: 1,
commentsCount: 0,
post_img: null,
isUserLiked: false,
usersLiked: [ 5edf43b93859680cf815e577 ],
exp_date: 2020-06-12T18:30:00.000Z,
has_img: false,
typology: 'chat',
geometry:
 { pintype: 'Point',
   _id: 5ee1f89732fd2c33bccfec56,
   coordinates: [Array] },
category: [],
created: 1591867543468,
createdAt: 2020-06-11T09:25:43.478Z,
updatedAt: 2020-06-15T10:01:01.133Z,
user_id:  { img: 'default-user-profile-image.png',
   _id: 5edd103214ce223088a59236,
   user_name: 'Puka' }
}

My userSchema is something like below

var userSchema = mongoose.Schema({
//some other fields
    user_name: {
        type: String,
        max: 30,
        min: 5
    },

    img: {
        type: String,
        default: 'default-user-profile-image.png'

    },
//some other fields
});
userSchema.plugin(uniqueValidator);
var userSchemaModel = mongoose.model('users', userSchema);

module.exports = {
    userSchemaModel,
}

According to the other answers here I tried using mongoose.Types.ObjectId(userId), but it gives complete empty set.

What can be the problem here and it will be really helpful if someone can help me with this as I'm stuck with this for days.

Update : post schema

var postSchema = mongoose.Schema({
    user_id: {
        type: String,
        required: true,
        ref: 'users'
    },
//other fields
var postSchemaModel = mongoose.model('posts', postSchema);
module.exports = {
    postSchemaModel,
}

user_1656749 · Answer

Since the data type of user._id(ObjectId) and post.user_id(String) are not the same you can not join those fields using $lookup. You have to make sure they are the same type before doing $lookup

If you are allowed to change the schema, it's recommended to use ObjectId for post.user_id

var postSchema = mongoose.Schema({
    user_id: {
        type: mongoose.Types.ObjectId,
        required: true,
        ref: 'users'
    },
    // ...other fields

But do remember to change the existing data type to ObjectId as well.

If you are really not allowed to change the schema and existing data, for some reason. You can convert the post.user_id to ObjectId in case that the data contains valid hexadecimal representation of ObjectId (available from MongoDB v4.0)

[
// prior pipeline stages
{ "$sort": { "createdAt": -1 } },
{
  "$addFields": {
    "user_id": { "$toObjectId": "$user_id" }
  }
},
{
  "$lookup": {
    "from": userSchemaModel.collection.name,
    "localField": "user_id",
    "foreignField": "_id",
    "as": "user_id"
  }
},
// other stages

user_4617496 · Answer

As the _id field in mongodb is stored as type ObjectId but in the posts collection user_id is stored as type string, therefore it is not able find the user information and bring blank array.

To resolve this save a plain string version of _id in user collection when a user is created. for example

{ 
  _id: ObjectId("5ee1f89732fd2c33bccfec55"),
  doc_id: "5ee1f89732fd2c33bccfec55",
  //other user info
}

and then use this doc_id field in $lookup

{
  "$lookup": {
      "from": userSchemaModel.collection.name,
      "localField": "user_id",
      "foreignField": "doc_id",
      "as": "user_id"
  }
}

In this way both user_id and doc_id will be of type string and will not need any type conversion hassles.

$lookup returns an empty array mongoose

2 Answers