I apologize if the question title is confusing; feel free to suggest an alternative.
I am writing some data analysis code and I have a set of data that I want to construct a map with, which can be used to look up certain keys to find certain values. But I want to be able to use a wild card for parts of the multidimensional key so that multiple values are returned. I also want fast lookup and efficient memory use.
My key is 5 dimensional. If I didn't want to be able to use wild cards in the lookup, then the solution is simply to use a 5 dimensional tuple for the key of a Map. Sometimes I do only want to return a single value using a fully specified key, but other times I want to return an aggregation of the results for a partially specified key. The value I am storing is a 3 dimensional tuple of floats.
If I didn’t want efficient memory usage, then I would create a 5 dimensional array and create 5 index lookups (one for each part of the key) then wrap the whole this with appropriate accessor methods. To get all values where one of the key parts is a wild card, simply access the array with ‘0..’, then aggregate the results as desired. In this case, much of the array will be empty. It doesn’t seem like a good idea because I don’t know in advance how big each dimension will be so the cross product could be beyond what is possible. The solution I have gone with for now is just to have separate maps for each kind of look up I am interested in. For instance, if my full key is k1*k2*k3*k4*k5, and part of my code wants to access aggregated data for anything that matches k1**k3*k4* then I will create a map indexed by k1*k3*k4 that stores the aggregated float it cares about. The number of combinations is rather large (32 * 3 = 96, I think) so if I wanted the ability to access this data in every possible way using the method I have gone with, I would need 96 different maps and accessor methods.
As I get to the end of writing this I realise that there must be a way of doing it because databases manage this problem just fine. How do they do it and can I do the same in F# in memory?