FsCheck: Override generator for a type, but only in the context of a single parent generator

user_2436737 0
Viewed 64

I seem to often run into cases where I want to generate some complex structure, but a special variation with a member type generated differently.

For example, consider this tree

type Tree<'LeafData,'INodeData> =
    | LeafNode of 'LeafData
    | InternalNode of 'INodeData * Tree<'LeafData,'INodeData> list

I want to generate cases like

  • No internal node is childless
  • There are no leaf-type nodes
  • Only a limited subset of leaf types are used

These are simple to do if I override all generation of a corresponding child type. The problem is that it seems register is inherently a thread-level action, and there is no gen-local alternative.

For example, what I want could look like

let limitedLeafs =
  gen {
    let leafGen = Arb.generate<LeafType> |> Gen.filter isAllowedLeaf
    do! registerContextualArb (leafGen |> Arb.fromGen)
    return! Arb.generate<Tree<NodeType, LeafType>>
  }

This Tree example specifically can work around with some creative type shuffling, but that's not always possible.

It's also possible to use some sort of recursive map that enforces assumptions, but that seems relatively complex if the above is possible. I might be misunderstanding the nature of FsCheck generators though.

Does anyone know how to accomplish this kind of gen-local override?

property-testing fscheck f#
2 Answers

There's a few options here - I'm assuming you're on FsCheck 2.x but keep scrolling for an option in FsCheck 3.

The first is the most natural one but is more work, which is to break down the generator explicitly to the level you need, and then put humpty dumpty together again. I.e don't rely on the type-based generator derivation so much - if I understand your example correctly that would mean implementing a recursive generator - relying on Arb.generate<LeafType> for the generic types.

Second option - Config has an Arbitrary field which you can use to override Arbitrary instances. These overrides will take effect even if the overridden types are part of the automatically generated ones. So as a sketch you could try:

Check.One ({Config.Quick with Arbitrary = [| typeof<MyLeafArbitrary>) |]) (fun safeTree -> ...)

More extensive example which uses FsCheck.Xunit's PropertyAttribute but the principle is the same, set on the Config instead.

Final option! :) In FsCheck 3 (prerelease) you can configure this via a new (as of yet undocumented) concept ArbMap which makes the map from type to Arbitrary instance explicit, instead of this static global nonsense in 2.x (my bad of course. seemed like a good idea at the time.) The implementation is here which may not tell you all that much - the idea is that you put an ArbMap instance together which contains your "safe" generators for the subparts, then you ArbMap.mergeWith that safe map with ArbMap.defaults (thus overriding the default generators with your safe ones, in the resulting ArbMap) and then you use ArbMap.arbitrary or ArbMap.generate with the resulting map.

Sorry for the long winded explanation - but all in all that should give you the best of both worlds - you can reuse the generic union type generator in FsCheck, while surgically overriding certain types in that context.

user_722110

FsCheck guidance on this is:

To define a generator that generates a subset of the normal range of values for an existing type, say all the even ints, it makes properties more readable if you define a single-case union case, and register a generator for the new type:

As an example, they suggest you could define arbitrary even integers like this:

type EvenInt = EvenInt of int with
    static member op_Explicit(EvenInt i) = i

type ArbitraryModifiers =
    static member EvenInt() = 
        Arb.from<int> 
        |> Arb.filter (fun i -> i % 2 = 0) 
        |> Arb.convert EvenInt int
        
Arb.register<ArbitraryModifiers>() |> ignore

You could then generate and test trees whose leaves are even integers like this:

let ``leaves are even`` (tree : Tree<EvenInt, string>) =
    let rec leaves = function
        | LeafNode leaf -> [leaf]
        | InternalNode (_, children) ->
            children |> List.collect leaves
    leaves tree
        |> Seq.forall (fun (EvenInt leaf) ->
            leaf % 2 = 0)

Check.Quick ``leaves are even``   // output: Ok, passed 100 tests.

To be honest, I like your idea of a "gen-local override" better, but I don't think FsCheck supports it.

user_3442230
Related