MongoDB find with $text index doesn't work properly

Question

I am trying to implement a search for a user using his username, username fields are unique. The result is to be all users whose username contains the string. During the query, the 'q' parameter is correctly passed to the find () function, but when the 'q' parameter contains only a part of the username in the response, an empty array is returned. However, when the whole username is given, the answer is correct.

My users.controller function:

async findAllUsers(req, res) {
    const sort_by = {};
        sort_by[req.query.sort_by || 'createdAt'] = req.query.order_by || 'desc';
        const offset = parseInt(req.query.offset) || 0;
        const per_page = parseInt(req.query.per_page) || 2;
        const usersPromise =
        User.find(req.filters)
            .skip(offset)
            .limit(per_page)
            .sort(sort_by);
        const countPromise = User.countDocuments(req.filters);
        const [users, count] = await Promise.all([usersPromise, countPromise]);
        return res.status(200).send({ data: users, count });
  }

My filter function:

export default function getFilters(req, res, next){
  const availableFilters = Object.keys(Users.schema.paths);
  const filters = qs.parse(req.query);

  const schemaFilters = _.pickBy(filters, (value, key) => availableFilters.indexOf(key) > -1);
  let searchFilter = {};
  if(filters.q){
    searchFilter = {
      $text: {
        $search: filters.q
      }
    }
  }

  req.filters = { ...searchFilter, ...schemaFilters };
  next(); 
}

My user model looks like this:

const User = new mongoose.Schema({
  username: {
    type: String,
    required: true,
    unique: true,
    lowercase: true,
    minlength: 6,
  },
  password:{
    type: String,
    required: true,
    minlength: 8
  },
  email:{
    type: String,
    unique: true,
    required: true
  },
  age: {
    type: Number,
  },
  description: String, 
  permissions: {
    type: Number,
    required: true,
    enum: [1, 2, 3],
  }
}, { timestamps: true });

User.index({ username: 'text' });

export default mongoose.model('User', User);

and my route looks like this:

  api.get("/users", getFilters, catchAsync(usersController.findAllUsers));

Example of user in db:

{
    "_id" : ObjectId("5fba87e118f11129905aea93"),
    "username" : "Brody_Muller",
    "password" : "$2b$10$Smb9HP5dIINvO0.6K48PkOTXpXn9Tu.H.218XRIHOMOUthrWNCk7C",
    "email" : "Mariane.Crist@gmail.com",
    "age" : 44,
    "description" : "Voluptas deserunt occaecati ut perferendis qui temporibus.",
    "permissions" : 3,
    "createdAt" : "2020-11-22T15:46:37.777Z",
    "updatedAt" : "2020-11-22T15:46:37.777Z"
}

I also tried to create an index in the describtion field, but here neither searching for part of string nor searching for whole string was unsuccessful. What am I doing wrong?

user_12582392 · Answer

Search does not work like that. It's probably more complicated than this, but it seems the only simple inferences are done.

"blue", it won't match blueberry; however, "blueberry" will match blueberries.

You can still create a standard index on the username, and then use regex, like so:

 db.test1.find({username:/Brody/})

This will match the query. You can improve things use insensitive search too, and making it the start of the word /^Brody/i.

Extended from comments. MongoDB on "stop words":

text indexes drop language-specific stop words (e.g. in English, the, an, a, and, etc.) and use simple language-specific suffix stemming.

Solution for this is:

db.coll.createIndex({description:"text"},{language:"none"})

If you specify a language value of "none", then the text index uses simple tokenization with no list of stop words and no stemming.

MongoDB find with $text index doesn't work properly

1 Answers