Nested arrays $unwind and $group back together in mongoDB

So the "rule" in restoring to original structure is that for each $unwind you did to "deconstruct" the document you now have to do a $group to restore it.

As you can imagine in such a pipeline this could be VERY cumbersome. but definitely doable. However let me propose a different approach that is still very messy but much easier compared to the alternative, additionally it is more efficient from a performance perspective.

(just minor sidenot the reason your score is still null is because you have a syntax error in your $multiply function)

Anyways, The idea is to first gather all the unique event ids that exist in the in nested documents. Then execute one lookup to fetch all the relevant events. And finally adding the score field using $map and $mergeDocuments instead of $unwinding and $grouping, like so:

Mongo Playground

db.collection.aggregate([
  {
    $addFields: {
      allEvents: {
        $reduce: {
          input: {
            $map: {
              input: "$principalCredits",
              in: {
                $map: {
                  input: "$$this.credits",
                  as: "credit",
                  in: {
                    $map: {
                      input: "$$credit.name.awardNominations.edges",
                      as: "edge",
                      in: "$$edge.node.award.event.id"
                    }
                  }
                }
              }
            }
          },
          initialValue: [],
          in: {
            "$concatArrays": [
              {
                "$reduce": {
                  input: "$$this",
                  initialValue: [],
                  in: {
                    "$concatArrays": [
                      "$$this",
                      "$$value"
                    ]
                  }
                }
              },
              "$$value"
            ]
          }
        }
      }
    }
  },
  {
    "$lookup": {
      "from": "eventsCollection",
      "localField": "allEvents",
      "foreignField": "id",
      "as": "matchingEvents"
    }
  },
  {
    $addFields: {
      principalCredits: {
        $map: {
          input: "$principalCredits",
          in: {
            $mergeObjects: [
              "$$this",
              {
                credits: {
                  $map: {
                    input: "$$this.credits",
                    as: "credit",
                    in: {
                      $mergeObjects: [
                        "$$credit",
                        {
                          name: {
                            "$mergeObjects": [
                              "$$credit.name",
                              {
                                "awardNominations": {
                                  "$mergeObjects": [
                                    "$$credit.name.awardNominations",
                                    {
                                      edges: {
                                        $map: {
                                          input: "$$credit.name.awardNominations.edges",
                                          as: "edge",
                                          in: {
                                            node: {
                                              $mergeObjects: [
                                                "$$edge.node",
                                                {
                                                  score: {
                                                    "$multiply": [
                                                      {
                                                        $cond: [
                                                          "$$edge.node.isWinner",
                                                          1.5,
                                                          1.2
                                                        ]
                                                      },
                                                      {
                                                        $first: {
                                                          $map: {
                                                            input: {
                                                              $filter: {
                                                                input: "$matchingEvents",
                                                                as: "matchedEvent",
                                                                cond: {
                                                                  $eq: [
                                                                    "$$matchedEvent.id",
                                                                    "$$edge.node.award.event.id"
                                                                  ]
                                                                }
                                                              }
                                                            },
                                                            as: "matched",
                                                            in: "$$matched.importance"
                                                          }
                                                        }
                                                      }
                                                    ]
                                                  }
                                                }
                                              ]
                                            }
                                          }
                                        }
                                      }
                                    }
                                  ]
                                }
                              }
                            ]
                          }
                        }
                      ]
                    }
                  }
                }
              }
            ]
          }
        }
      }
    }
  },
  {
    $unset: [
      "allEvents",
      "matchingEvents"
    ]
  }
])

Mongo Playground

I will just mention that you can make this much much much cleaner by involving some code while keeping the same approach suggested. first getting unique eventid with distinct. then fetching the matching importance for each event. Finally execute a single query using arrayFilters you can construct with this information.

• Final side not is that the provided pipeline did not deal with null or missing values. So if an array is missing an error will be thrown as $map expects input to be a valid array. This can easily be solved by just wrapping each of these expressions with $ifNull, like so:

{
    $map: {
        input: {$ifNull: ["$$this.credits",[]]}
    }
}

This will also replace null values with an empty []

The deep buried keys (...award.event.id) in arrays confounds an easy approach without 1) messing up the structure as the OP has noted 2) incurring potentially very expensive multiple $unwind calls.

Recommendation: Two pass approach. Get the necessary importance values for the principalCredits objects in question, then go back and manually iterate over the collection, diving into the structure and applying the logic score = importance * isWinner? 1.2 : 1.5

PASS 1: Get the ev data

c=db.foo.aggregate([
    {$project: {
        XX: {$reduce: {
            // Rapidly get to things we need to lookup:                                                
            input: '$principalCredits.credits.name.awardNominations.edges.node.award.event.id',
            // We end up with a mess incl. empty arrays...                                             
            //   [ [[ev1,ev2], [ev3,ev4]], [], [[ev1,...], []  ... ] ]                                 
            // Need to collapse all those arrays of arrays of arrays into                              
            // a single list of ev values, hence a reduce within a reduce:                             
            initialValue: [],
            in: {$concatArrays: [
                '$$value',
                {$reduce: {
                    input: '$$this',
                    initialValue: [],
                        in: {$concatArrays: [ '$$value', '$$this' ] }
                }} ]}
        }}
    }}

    // XX is now [ ev1,ev2,ev3,ev4,ev1 ... ]                                                           
    // The empty arrays are ignored.  Don't worry about dupes.                                         
    ,{$lookup: {
        from: "Xev",
        let: { evids: "$XX" },
        pipeline: [
            {$match: {$expr: {$in: ["$id","$$evids"]} } }
        ],
        as: 'XX' // overwrite XX...                                                                    
    }}
]);

evdict = {}
c.forEach(function(d) {
    d['XX'].forEach(function(ww) {
        evdict[ww['id']] = ww;
    });
});

{
    "ev0003786" : {
        "_id" : ObjectId("62cd7f8138d0fbc0eacfb17f"),
        "id" : "ev0003786",
        "text" : "Millennium Docs Against Gravity",
        "importance" : 1
    },
    "ev0000351" : {
        "_id" : ObjectId("62cd7f8138d0fbc0eacfb180"),
        "id" : "ev0000351",
        "text" : "International Documentary Association",
        "importance" : 2
    },
    "ev0000571" : {
        "_id" : ObjectId("62cd7f8138d0fbc0eacfb181"),
        "id" : "ev0000571",
        "text" : "Royal Television Society, UK",
        "importance" : 3
    }
}

PASS 2: Iterate main collection Left as exercise to reader.

Note that if

1. The number of events is small.
2. There is no need or value in performing $match on the initial principalCredits collection (i.e. before the fancy $project/$reduce) to significantly reduce the lookup set into events

then this whole thing is unnecessary. Simply slurp all events into evdict with a quick find and proceed to pass 2.

There is potentially a very cool solution that can do this in one pass

UPDATED

See Tom's answer below. Note to MongoDB 5.0 users: The new $getField function allows you to pluck out fields by name instead of having to use the standard trick of using dot notation in the $in clause to access the field. This might be clearer to some:

{$getField: {
    "field": "importance",
    "input": {
      $first: {
        $filter: {
          input: "$matchingEvents",
          as: "matchedEvent",
          cond: {
            $eq: [
              "$$matchedEvent.id",
              "$$edge.node.award.event.id"
            ]
          }
        }
      }
    }
  }
}