F#: What to call a combination of map and fold, or of map and reduce?

Viewed 150

A simple example, inspired by this question:

module SimpleExample =
    let fooFold projection folder state source =
        source |> List.map projection |> List.fold folder state
    // val fooFold :
    //   projection:('a -> 'b) ->
    //     folder:('c -> 'b -> 'c) -> state:'c -> source:'a list -> 'c

    let fooReduce projection reducer source =
        source |> List.map projection |> List.reduce reducer
    // val fooReduce :
    //   projection:('a -> 'b) -> reducer:('b -> 'b -> 'b) -> source:'a list -> 'b

    let game = [0, 5; 10, 15]
    let minX, maxX = fooReduce fst min game, fooReduce fst max game
    let minY, maxY = fooReduce snd min game, fooReduce snd max game

What would be a natural name for the functions fooFold and fooReduce in this example? Alas, mapFold and mapReduce are already taken.

mapFold is part of the F# library and does a fold operation over the input to return a tuple of 'result list * 'state, similar to scan, but without the initial state and the need to provide the tuple as part of the state yourself. Its signature is:

val mapFold : ('State -> 'T -> 'Result * 'State) -> 'State -> 'T list -> 'Result list * 'State

Since the projection can easily be integrated into the folder, the fooFold function is only included for illustration purposes.

And MapReduce:

MapReduce is an algorithm for processing huge datasets on certain kinds of distributable problems using a large number of nodes


Now for a more complex example, where the fold/reduce is not directly applied to the input, but to the groupings following a selection of the keys. The example has been borrowed from a Python library, where it is called - perhaps misleadingly - reduceby.

module ComplexExample =
    let fooFold keySelection folder state source =
        source |> Seq.groupBy keySelection 
        |> Seq.map (fun (k, xs) ->
            k, Seq.fold folder state xs) 
    // val fooFold :
    //   keySelection:('a -> 'b) ->
    //     folder:('c -> 'a -> 'c) -> state:'c -> source:seq<'a> -> seq<'b * 'c>
    //     when 'b : equality

    let fooReduce keySelection projection reducer source =
        source |> Seq.groupBy keySelection 
        |> Seq.map (fun (k, xs) ->
            k, xs |> Seq.map projection |> Seq.reduce reducer) 
    // val fooReduce :
    //   keySelection:('a -> 'b) ->
    //     projection:('a -> 'c) ->
    //     reducer:('c -> 'c -> 'c) -> source:seq<'a> -> seq<'b * 'c>
    //     when 'b : equality

    type Project = { name : string; state : string; cost : decimal }
    let projects =
        [ { name = "build roads";  state = "CA"; cost = 1000000M }
          { name = "fight crime";  state = "IL"; cost = 100000M  }
          { name = "help farmers"; state = "IL"; cost = 2000000M }
          { name = "help farmers"; state = "CA"; cost = 200000M  } ]
    fooFold (fun x -> x.state) (fun acc x -> acc + x.cost) 0M projects
    // val it : seq<string * decimal> = seq [("CA", 1200000M); ("IL", 2100000M)]

    fooReduce (fun x -> x.state) (fun x -> x.cost) (+) projects
    // val it : seq<string * decimal> = seq [("CA", 1200000M); ("IL", 2100000M)]

What would be the natural name for the functions fooFold and fooReduce here?

2 Answers

I'd probably call the first two mapAndFold and mapAndReduce (though I agree that mapFold and mapReduce would be good names if they were not already taken). Alternatively, I'd go with mapThenFold (etc.), which is perhaps more explicit, but it reads a bit cumbersome.

For the more complex ones, reduceBy and foldBy sound good. The issue is that this would not work if you also wanted a version of those functions that do not do the mapping operation. If you wanted that, you'd probably need mapAndFoldBy and mapAndReduceBy (as well as just foldBy and reduceBy). This gets a bit ugly, but I'm afraid that's the best you can do.

More generally, the issue when comparing names with Python is that Python allows overloading whereas F# functions do not. This means that you need to have a unique name for functions that would have multiple overloads. This means that you just need to come up with a consistent naming scheme that will not make the names unbearably long.

(I experienced this when coming up with names for the functions in the Deedle library, which is somewhat inspired by Pandas. You can see for example the aggregation functions in Deedle for an example - there is a pattern in the naming to deal with the fact that each function needs a unique name.)

I have a different opinion as Thomas.

First; I think that not having overloads is a good thing, and giving every operation unique names is also something good. I also would say that giving long names to functions rarely used is even more important and should not be avoided.

Writing longer names is usally never a problem as we as programers usually use an IDE with auto-completion. But reading and understanding is different. Knowing what a functions does because of a long descriptive name is better then a short name.

A long descriptive function name gets more important the less often a function is used. It helps reading and understanding the code. A short and less descriptive function name that is rarely used causes confusion. The confusion would just increase if it even would be just an overload of another function name.

Yes; naming things can be hard, that's the reason why its important and shoudn't be avoided.


To what you describe. I would have name it mapFold and mapReduce. As those exactly describe what they do.

There is already a mapFold in F#, and in my opinion, the F# devs fucked up either with the naming, arguments or the output of the function. But anyhow, they just fucked up.

I usually would have expected mapFold to do map and then fold. Actually it does, but it also returns the intermediate list that is created on the run. Something I would not expect it to return. And i would also expect it to pass two functions instead of one.

When we get to Thomas suggestion on naming it mapAndFold or mapThenFold. Then i would expect different behaviour for those two functions. mapThenFold exactly tells what it does. map and then fold on it. I think the then is not important. That's also why I would name it mapFold or mapReduce. Writing it this way already suggest a then.

But mapAndFold or mapAndReduce does not tell something about the order of execution. It just says it does two things or somehow returns this AND that.

With that in mind, i would say that the F# library should have named its mapFold either mapAndFold, changed the return value to just return the fold (and have two arguments instead of one). But hey, its fucked up now, we cannot change it anymore.

As for mapReduce, I think you are a little bit mistaken. The mapReduce algorithm is named that way, because it just does map and then reduce. And that's it.

But functional programming with its stateless and more descriptive operations sometimes have additional benefits. Technically a map is less powerful compared to a for/fold as it just describes how values are changed, without that the order matters or the position in a list. But because of this limitation, you can run it in parallel, even on a big computer cluster. And that's all what mapReduce Algorithm you cite do.

But that doesn't mean a mapReduce must always run its operation on a big cluster or in parallel. In my opinion you could just name it mapReduce and that's fine. Everybody will know what it does and I think nobody expect it to suddenly run on cluster.

In general I think the mapFold that F# provides is silly, here are 4 examples how I think it should have been provided.

let double x = x * 2
let add x y  = x + y

mapFold      double add 0 [1..10] // 110
mapAndFold   double add 0 [1..10] // [2;4;6;8;10;12;14;16;18;20] * 110
mapReduce    double add   [1..10] // Some (110)
mapAndReduce double add   [1..10] // Some ([2;4;6;8;10;12;14;16;18;20] * 110)

Well mapFold doesn't work that way, so you have the following options.

  1. Implement mapReduce the way you have it. And ignore the in-consistency with mapFold.
  2. Provide mapAndReduce and mapReduce.
  3. Make your mapReduce return the same crap as the default implementation of mapFold does and provide mapThenReduce.
  4. Like (3) but also add mapThenFold.

Option 4 has the most compatibility and expectation of what already exists in F#. But that doesn't mean you must do it that way.

In my opinion I would just:

  1. implement mapReduce returning the result of map and then reduce.
  2. I wouldn't care about a mapAndReduce version that returns a list and the result.
  3. Provide a mapThenFold expecting two function arguments returning the result just of fold.

As a general notice: Implementing mapReduce just by calling map and then reduce is somewhat pointless. I would expect it to have a more low-level implementation that does both things by just traversing the data-structure once. If not, i just can call map and then reduce anyway.

So an implementation should look like:

let mapReduce mapper reducer xs =
    let rec loop state xs =
        match xs with
        | []    -> state
        | x::xs -> loop (reducer state (mapper x)) xs
    match xs with
    | []    -> ValueNone
    | [x]   -> ValueSome (mapper x)
    | x::xs -> ValueSome (loop (mapper x) xs)

let double x = x * 2
let add x y  = x + y

let some110 = mapReduce double add [1..10]
Related