Cosmos SQL query to flatten all nested properties with a given name

Christian Smith 0
Viewed 35

We persist Cosmos documents with a tree of nodes that can have an unbounded depth, e.g. children have arrays of children who have arrays of children etc etc.

We need to query the document and flatten all properties of a given name (queryRef) from all levels in the tree.

This is the persisted document:

{
    id: "1",
    children: [
        {
            id: "2",
            queryRef: {ref: "a"},
            children: [
                {
                    id: "3",
                    queryRef: {ref: "b"},
                    children: [
                        {
                            id: "5",
                            queryRef: {ref: "c"},
                            children: [
                                ...
                            ]
                        }
                    ]
                },
                ...
            ]
        }
    ]
}

This is the desired response:

{
    id: "1",
    queryRefs: [
        {
            ref: "a",
        },
        {
            ref: "b",
        },
        {
            ref: "c",
        }   
    ]
}

With a query along the lines of:

SELECT c.id, c.children[...].queryRef FROM c
azure-cosmosdb-sqlapi azure-cosmosdb
1 Answers

While Cosmos DB's query engine is designed to work on hierarchical data, it does not provide the capability to write queries that recursively traverses arrays within objects of unknown depth then project out flat as you've defined. You need to tell the query engine the structure in which to look and project back out.

Another issue here too is the structure you've defined does not follow the general rules of when to embed vs when to reference. Unbounded arrays should be modeled as individual documents rather than embedded within a single document. Doing so provides multiple benefits.

  1. It ensures that as the amount of data grows, the performance and cost of operations on that data has consistent latency and remains efficient. As the size of a document grows, the cost of operations on it grows in a non-linear fasion.
  2. Inserting these children as individual documents allows you to query the data using generally simple queries.
  3. It avoids the potential your document hits the 2MB limit for a document in Cosmos DB.

You certainly can have tree-like structures within your documents and modeling one-to-few relationships is definitely fine for embedding within a single document. But whether a simple query or more complex where you need to traverse its structure, you need to have a defined model that allows you to identify individual objects to both project and apply filter predicates on the data.

userHeI30
Related