How to get detailed explanations when a document fails schema validation in mongodb 5.0?

user_9928977 0
Viewed 480

MongoDB 5.0 adds detailed explanations when a document fails schema validation.

Here is an example of validation error with details: https://docs.mongodb.com/manual/core/schema-validation/#existing-documents

"details" : [
                  {
                    "operatorName" : "bsonType",
                    "specifiedAs" : {
                       "bsonType" : "string"
                        },
                    "reason" : "type did not match",
                    "consideredValue" : 10,
                    "consideredType" : "double"
                  }

But I can't get any detailed error messagea on my server.

mongod version: v5.0.2

Here is a simple example on Python:

from pymongo import MongoClient

client = MongoClient("mongodb://localhost:27017")

db = client["pymongo"]

schema = {"$jsonSchema": {"required": ["name", "value"]}}

col = db["t0"]
col.drop()

col = db.create_collection("t0", validator=schema)

col.insert_one({"_id": 1, "name": "n1", "value": 2})
# insert ok

col.insert_one({"_id": 2, "name": "n2"})
# this fails with the error

Here is the error, no details:

pymongo.errors.WriteError: Document failed validation, full error: {'index': 0, 'code': 121, 'errmsg': 'Document failed validation'}

I tried insert such invalid document in mongo cli client, in MongoDB Compass. There are errors without details again.

This my database was upgraded from 4.4 version. Maybe it's necessary to turn on some setting on my cluster to show detailed error messages for schema validation?

mongodb
2 Answers

The 5.0 server provides these validation messages in the responses. If you issue the write commands manually using your driver's command helper (e.g. see here for Ruby) you'll get the raw response back and you can retrieve the details from there.

Naturally most people use driver methods to write to the database, and the driver counterpart to this server feature is this one which is currently being worked on. Follow the links to the individual driver tickets, for pymongo this is https://jira.mongodb.org/browse/PYTHON-2553 which is fixed in 3.12 (released) and 4.0 (unreleased). So you need to update your driver.

See this for usage example.

Maybe this feature requires FCV to be 5.0 also, test a brand new 5.0 deployment to verify and if so upgrade fully to 5.0.

user_95409250

I tried your example but with MongoDB 5.0-ent. Your example does not show any authentication, but when I try with the following...

from pymongo import MongoClient

client = MongoClient("mongodb://barry:barry@localhost:50001")                                                                                                                                                                                                             

db = client["pymongo"]

schema = {"$jsonSchema": {"required": ["name", "value"]}}

col = db["t0"]
col.drop()

col = db.create_collection("t0", validator=schema)

col.insert_one({"_id": 1, "name": "n1", "value": 2}) 
# insert ok

col.insert_one({"_id": 2, "name": "n2"})
# this fails with the error

I get the following response...

pythontest % python3 test.py
Traceback (most recent call last):
  File "/Users/barron.anderson/junk/pythontest/test.py", line 17, in <module>
    col.insert_one({"_id": 2, "name": "n2"})
  File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 698, in insert_one
    self._insert(document,
  File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 613, in _insert
    return self._insert_one(
  File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 602, in _insert_one
    self.__database.client._retryable_write(
  File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1498, in _retryable_write
    return self._retry_with_session(retryable, func, s, None)
  File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1384, in _retry_with_session
    return self._retry_internal(retryable, func, session, bulk)
  File "/usr/local/lib/python3.9/site-packages/pymongo/mongo_client.py", line 1416, in _retry_internal
    return func(session, sock_info, retryable)
  File "/usr/local/lib/python3.9/site-packages/pymongo/collection.py", line 600, in _insert_command
    _check_write_command_response(result)
  File "/usr/local/lib/python3.9/site-packages/pymongo/helpers.py", line 226, in _check_write_command_response
    _raise_last_write_error(write_errors)
  File "/usr/local/lib/python3.9/site-packages/pymongo/helpers.py", line 208, in _raise_last_write_error
    raise WriteError(error.get("errmsg"), error.get("code"), error)
pymongo.errors.WriteError: Document failed validation, full error: {'index': 0, 'code': 121, 'errInfo': {'failingDocumentId': 2, 'details': {'operatorName': '$jsonSchema', 'schemaRulesNotSatisfied': [{'operatorName': 'required', 'specifiedAs': {'required': ['name', 'value']}, 'missingProperties': ['value']}]}}, 'errmsg': 'Document failed validation'}
user_4155590
Related