MongoDB has been a really great fit for an application I'm trying to build, except for potentially one area which I am REALLY hoping there's a solution to. The majority of the data will be a forest. For a given node (or rather, document) in a tree in the forest, I would like to return all ancestors up to the root of that tree.
Because of Mongo's distributed nature, documents belonging to the same tree may not reside on the same node and thus, the most intuitive way of getting this information (via recursion), may be costly. I've read that we can control which nodes hold which documents via the shard key, so I have thought that perhaps each document in the tree can hold a reference to the ObjectId of the root document of the tree and use that as the shard key--this way I can guarantee all documents of a particular tree can reside on a single node, yet the entire forest will be distributed amongst the nodes in the sharded cluster. However, even if this is the case, I don't know enough about the mongos to know that it will indeed forward the recursive request to the single node that contains the tree and perform the recursive query on that node and that node only--so that is my first question, is it possible to delegate the recursive query to a single node that you know contains all documents that'd be returned in such a query? If so, then perhaps the recursive query won't be too terrible.
The above approach poses a problem though, what if some particular tree grows very large? I can't think of any other way of handling this problem other than moving the tree to a dedicated node for such purposes, possibly using some heuristic to preemptively relocate a tree before it gets too large (when it'd be even mode expensive to transfer). However, I don't know if that's even possible in MongoDB--to relocate data from one cluster node to another--and I don't know if I could have that kind of control of sharding logic on the tree that has been moved to the dedicated node, something along the lines of:
if shard_key in special_shard_keys:
store data in dedicated node
else:
store data in node using other sharding logic
If above logic is possible, then it solves another concern: not all documents should be sharded the exact same way. Aside from the trees I'll be storing, I'll store other non-tree like structures, it would be nice if these could have their own dedicated sharding logic based on other things, so my last question is then: is it possible to have per document type sharding logic?