Extensible Records and Higher Order Functions

Your reasoning about what you know about a WithId a value and what you should therefore be able to do with it (i.e. treat it as an a) is logically sound, but Elm’s support for extensible records just doesn't play with the type system in a way that permits that. Indeed, Elm’s extensible record types are specifically designed not to allow access to the full record.

What extensible records are designed for is to declare that a function will only access certain fields of a record, and the compiler then enforces that constraint:

type alias WithId a =
    { a | id : Uuid }

isValidId : WithId a -> Bool
isValidId withId =
    -- can only access .id

updateId : WithId a -> WithId a
updateId withId =
    { withId |
        id = -- can only access .id
    }

When used to write functions that work on a small slice of a large program model, for example, this provides powerful guarantees about what a given function is able to do. Your updateUser function, for example, can be constrained only to access the user field of your model, rather than having free reign to access perhaps dozens of fields in your model. When you are later trying to debug an unexpected change to your model, these constraints help you quickly rule out many functions as being responsible for that change.

For more concrete examples of the above, I recommend Richard Feldman’s elm-europe 2017 talk https://youtu.be/DoA4Txr4GUs (see 23:10 in that video for the part showing extensible records).

I had the same excitement followed by confused disappointment when I first learned about extensible records, but in hindsight I think that comes from my deeply ingrained Object Oriented Programming instincts. I saw “extensible” and my brain jumped straight to “inheritance”. But Elm doesn't implement extensible records in a way that can support type inheritance.

The real intended purpose of extensible records, as illustrated in the examples above, is to constrain functions’ access to only a subset of a large record type.

As for your specific use case, I'd suggest starting with an Encoder for each of your data types, and if you want to reuse the implementation of an ID encoder, call a shared idEncoder function from within each of your encoders.

The only thing we know about an instance of WithId ais that is has a .id field. If we encode a WithId a the best we can do is encode the .id value.

type alias Uuid =
    String

encodeUuid : Uuid -> Value
encodeUuid =
    Encode.string

type alias WithId a =
    { a | id : Uuid }

encode : WithId a -> Value
encode withId =
    encodeUuid withId.id

Reading your addenda and re-reading your question, I believe you actually can achieve what you want with extensible records in Elm.

Here's a little program that does what you're after and compiles just fine:

module Main exposing (main)

import Html exposing (Html, pre, text)
import Json.Encode as Json exposing (Value, int, object, string)


main : Html msg
main =
    let
        json =
            Json.encode 2 <|
                encoder recordEncoder record
    in
    pre [] [ text json ]


type alias WithId a =
    { a | id : Int }


type alias Record =
    WithId
        { fieldA : String
        }


record : Record
record =
    { id = 123
    , fieldA = "A"
    }


encoder : (WithId a -> Value) -> WithId a -> Value
encoder recordEncoder value =
    object
        [ ( "id", int value.id )
        , ( "value", recordEncoder value )
        ]


recordEncoder : Record -> Value
recordEncoder record =
    object
        [ ( "fieldA", string record.fieldA ) ]

View on Ellie - The Elm Live Editor

As per my first answer I'm still pretty sure it won't work out well if you extend this pattern to your whole program, but the nice thing about Elm is that you can try stuff like this and if it doesn't work out, the compiler is there to help you refactor!

…and if you really need to be able to deal with both persisted and non-persisted records in your program, you can do it with just a little repetition in your type aliases:

module Main exposing (main)

import Html exposing (Html, pre, text)
import Json.Encode as Json exposing (Value, int, object, string)


main : Html msg
main =
    let
        json =
            Json.encode 2 <|
                encoder recordEncoder record
    in
    pre [] [ text json ]


type alias WithId a =
    { a
        | id : Int
    }


type alias Record a =
    { a
        | fieldA : String
    }


type alias PersistedRecord =
    { id : Int
    , fieldA : String
    }


record : PersistedRecord
record =
    { id = 123
    , fieldA = "A"
    }


encoder : (WithId a -> Value) -> WithId a -> Value
encoder recordEncoder value =
    object
        [ ( "id", int value.id )
        , ( "value", recordEncoder value )
        ]


recordEncoder : Record a -> Value
recordEncoder record =
    object
        [ ( "fieldA", string record.fieldA ) ]

View on Ellie - The Elm Live Editor

To answer this part of your addenda directly:

I guess I don't understand why we say:

type alias Foo = { bar : Int } 

instead of

type Foo = { bar : Int }

The former implies to me that any record with { bar : Int } is a Foo, which would imply to me that {a|bar:Int} is both of the same type as a and a Foo. Where am I wrong here? I'm confused. I feel like I'm not grokking records.

{ a | bar:Int } is not a Foo, because the Foo type alias specifies a fixed set of fields. Foo contains a bar field and no other fields. If you want to say “a record with these fields and possibly others”, you need to use the extensible record syntax ({ a | …fields… }).

So I think where you're going wrong is in assuming that all record types are extensible. They aren't. When a function says it takes or returns a record with a fixed set of fields, those values will never contain other fields—the compiler guarantees it.