Parsing of collection validator failed :: caused by :: Unknown $jsonSchema keyword: examples in MongoDB/Ajv/Typebox

user_3977826 0
Viewed 115

I'm using an Atlas based mongodb instance and fastify for backend. I'm also using @sinclair/typebox to generate JSON Schemas for data validation. Normally, I use these schemas to validate input messages but I now want to reuse them as validators for MongoDB collections too.

I have something like this:

    await database.command({
      collMod: 'users',
      validator: {
        $jsonSchema: UserSchema,
      },
    });

And also, I have examples set for some of the fields. e.g.:

export const HandleSchema = Type.RegEx(/^[a-zA-Z0-9_-]{1,24}$/, {
  examples: ['harry-potter', 'jane-doe-99'],
  title: 'Handle',
  // ...
});

export const UserSchema = Type.Object(
  {
    _id: HandleSchema,
    // ...
  },
  { additionalProperties: false },
);

However, I'm getting this error while trying to apply this validator:

Parsing of collection validator failed :: caused by :: Unknown $jsonSchema keyword: examples

Which means the examples keyword is not recognized by MongoDB; and unfortunately looks like this happens in server side. Looks like they dared to ignore the behavior suggested by the spec. I'm also seeing a flag that can be used to ignore unknown keywords, but unfortunately, setParameter doesn't work with atlas.

Now, at least I should be able to remove examples somehow from typebox or ajv. I tried Type.Strict(UserSchema) and ajv.customOptions.keywords set to ['examples'] (this probably is the inverse of what I want to achieve), but they didn't work.

How do I ignore examples field just for the mongodb schema? Or how do I fix this from MongoDB Atlas side (looks impossible)? Any other approach?

Thanks in advance.

ajv fastify mongodb-atlas node.js mongodb
2 Answers

Regardless of how much I dislike this approach, I went for a "temporary" solution.

Since the end result of a json schema -- as mongodb sees it -- is nothing more than a plain old javascript object. So I wrote a utility function to recursively strip-out the unnecessary fields.

import { TObject, TProperties } from '@sinclair/typebox';

export function stripExtras<T extends TProperties>(
  obj: TObject<T>,
  exclude: string[],
): TObject<T> {
  const result: { [k: string | symbol]: any } = {};

  for (const key in obj) {
    if (exclude.includes(key)) {
      continue;
    }

    if (typeof obj[key] === 'object' && !Array.isArray(obj[key])) {
      result[key] = stripExtras(obj[key], exclude);
      continue;
    }

    if (
      Array.isArray(obj[key]) &&
      obj[key].length > 0 &&
      typeof obj[key][0] === 'object'
    ) {
      result[key] = obj[key].map((item: TObject<TProperties>) =>
        stripExtras(item, exclude),
      );
      continue;
    }

    result[key] = obj[key];
  }

  return result as Exclude<TObject<T>, typeof exclude[number]>;
}

This is not completely accurate, especially the type definitions; I'll keep improving it. However, it does what I want.

const UserSchemaStripped = stripExtras(UserSchema, [
  'examples',
  'const',
  'default',
]);

Looks like examples was not the only field that mongodb reject.

And I'm using the stripped version only for the MondoDB.

   await database.command({
      collMod: 'users',
      validator: {
        $jsonSchema: UserSchemaStripped,
      },
    });
user_39778260

I am in the process of doing something very similar to what you are doing. Have you tried using an Unsafe type. As described in the documents:

Use Type.Unsafe(...) to create custom schemas with user defined inference rules.

So in your case you could so something like this:

import { Type } from '@sinclair/typebox';
import type { Static, TSchema } from '@sinclair/typebox';

function RegExType<T extends TSchema>(schema: T) {
    const { examples, ...rest } = schema;
    return Type.Unsafe<Static<T>>({ ...rest });
}

and then you can use it like this:

const HandleSchema = RegExType(
    Type.RegEx(/^[a-zA-Z0-9_-]{1,24}$/, {
        examples: ['harry-potter', 'jane-doe-99'],
        title: 'Handle',
        // ...
    })
);

export const UserSchema = Type.Object(
    {
        _id: HandleSchema,
    },
    { additionalProperties: false },
);

/** UserSchema object looks like this:
{
    "additionalProperties": false,
    "type": "object",
    "properties": {
        "id": {
            "title": "Handle",
            "type": "string",
            "pattern": "^[a-zA-Z0-9_-]{1,24}$"
        }
    },
    "required": [
        "id"
    ]
}
*/

Now I am not 100% certain this is the correct way of handling things as I'm still learning the library, but by the looks of it, it seems to do what you described. Unless I misunderstood your question.

You said you wanted to reuse the schema both for ajv and mongodb. I am not sure if you can use the exact same schema object to do both, but what you can easily do is make a wrapper function that outputs 2 schemas, for ajv and mongodb, depending on some parameter you pass to it.

Personally I think that would be a better implementation than writing a recursive function that strips out unnecessary keys.

user_32582510
Related