mongodb index is created but is not used during query

Viewed 278

Hope this is not a totally stupid question. I'm creating a compound index on mongodb on fields that contain strings. After running the index creation I do not see a process that creates the index via db.currentOp(), But I do see the index if I run getIndexes().

db.sampledb.createIndex(
    {
        "DevIdent.ParametersExt. ID": 1,
        "DevIdent.Parameters.Type": 1,
        "MetaData.SessionName": 1
    },
    {
        background: false,
        name: "sample",
    }
)

Running this query with explain I always get COLLSCAN I never see an index scan instead of COLLSCAN:

db.sampledb.aggregate([
    {
        "$match": {
            "DevIdent.ParametersExt.ID": { "$regex": ".*22~44.*" },
            "DevIdent.Parameters.Type": { "$ne": "TYPICAL" },
            "MetaData.SessionName": "2021_02_09_13_31_03"
        }
    }
])

am I missing something basic ?

3 Answers

This matching filter is the problem:

"DevIdent.ParametersExt.ID": { "$regex": ".*22~44.*" }

You are searching for the text 22~44 anywhere in the DevIdent.ParametersExt.ID field. The problem is that a B-tree based index can only be used if searching can start at the very beginning of the field, which is not the case here.

However, there is no reason why the index could not be used on the other two fields. But, since you started your multi-column index on a field which can't be used, therefore Mongo is choosing to just not use this index at all.

You could consider just defining an index on the other two fields:

db.sampledb.createIndex({
    "DevIdent.Parameters.Type": 1,
    "MetaData.SessionName": 1
},
{
    background:false,
    name: "sample",
});

Assuming that these two fields have high cardinality (i.e. they are very restrictive), then Mongo may choose to use the index, despite the regex condition on the third field.

An index can be used only for "prefixed" ^.... regexps. I find the documentation at https://docs.mongodb.com/manual/reference/operator/query/regex/#index-use is pretty clear about it so I will just quote it here:

For case sensitive regular expression queries, if an index exists for the field, then MongoDB matches the regular expression against the values in the index, which can be faster than a collection scan. Further optimization can occur if the regular expression is a “prefix expression”, which means that all potential matches start with the same string. This allows MongoDB to construct a “range” from that prefix and only match against those values from the index that fall within that range.

A regular expression is a “prefix expression” if it starts with a caret (^) or a left anchor (\A), followed by a string of simple symbols. For example, the regex /^abc.*/ will be optimized by matching only against the values from the index that start with abc.

Additionally, while /^a/, /^a./, and /^a.$/ match equivalent strings, they have different performance characteristics. All of these expressions use an index if an appropriate index exists; however, /^a./, and /^a.$/ are slower. /^a/ can stop scanning after matching the prefix.

Whatever everyone has posted as an answer holds true for inefficient use of index, but it still should come across as IXSCAN in explain plan, therefore I did a test.

db.sampledb.createIndex(
    {
        "DevIdent.ParametersExt. ID": 1,
        "DevIdent.Parameters.Type": 1,
        "MetaData.SessionName": 1
    },
    {
        background: false,
        name: "sample",
    }
)

This is your command there is some space between DevIdent.ParametersExt. & ID, which is causing an issue but you still won't be able to create a new index without the space, gives duplicate error. Remove this index all together and create a new one without the space.

EDIT : Schoolkid mistake, remove the name part, you should be able to create both the indexes, in the tree it is simply field name which is a string being stored with respective BSON types, therefore with the space and without it are two indexes separately.

db.sampledb.createIndex(
    {
        "DevIdent.ParametersExt.ID": 1,
        "DevIdent.Parameters.Type": 1,
        "MetaData.SessionName": 1
    },
    {
        background: false,
        name: "sample",
    }
)  

Try your aggregate command now, works for me.

{
"queryPlanner" : {
    "plannerVersion" : 1,
    "namespace" : "test_content.test4",
    "indexFilterSet" : false,
    "parsedQuery" : {
        "$and" : [ 
            {
                "MetaData.SessionName" : {
                    "$eq" : "2021_02_09_13_31_03"
                }
            }, 
            {
                "DevIdent.ParametersExt.ID" : {
                    "$regex" : ".*22~44.*"
                }
            }, 
            {
                "DevIdent.Parameters.Type" : {
                    "$not" : {
                        "$eq" : "TYPICAL"
                    }
                }
            }
        ]
    },
    "optimizedPipeline" : true,
    "winningPlan" : {
        "stage" : "FETCH",
        "inputStage" : {
            "stage" : "IXSCAN",
            "filter" : {
                "DevIdent.ParametersExt.ID" : {
                    "$regex" : ".*22~44.*"
                }
            },
            "keyPattern" : {
                "DevIdent.ParametersExt.ID" : 1.0,
                "DevIdent.Parameters.Type" : 1.0,
                "MetaData.SessionName" : 1.0
            },
            "indexName" : "sample",
            "isMultiKey" : false,
            "multiKeyPaths" : {
                "DevIdent.ParametersExt.ID" : [],
                "DevIdent.Parameters.Type" : [],
                "MetaData.SessionName" : []
            },
            "isUnique" : false,
            "isSparse" : false,
            "isPartial" : false,
            "indexVersion" : 2,
            "direction" : "forward",
            "indexBounds" : {
                "DevIdent.ParametersExt.ID" : [ 
                    "[\"\", {})", 
                    "[/.*22~44.*/, /.*22~44.*/]"
                ],
                "DevIdent.Parameters.Type" : [ 
                    "[MinKey, \"TYPICAL\")", 
                    "(\"TYPICAL\", MaxKey]"
                ],
                "MetaData.SessionName" : [ 
                    "[\"2021_02_09_13_31_03\", \"2021_02_09_13_31_03\"]"
                ]
            }
        }
    },
Related