How do I asynchronously download a web page in F# and catch errors

user_1769 0
Viewed 146

I've been trying to figure out the idiomatic F# way to download a webpage asynchronously and handle any errors/HTTP failure codes, I think this is my closest attempt so far but I'm getting a type error on the Choice1Of2 line

I'd love to

a) Understand why this fails (I'm still learning F#)
b) Know if this is the right approach/how to make this work or if I'm completely on the wrong track here

FS0001    This expression was expected to have type
    'Async<Choice<string,exn>>'    
but here has type
    'Choice<'a,'b>'

    let fetchAsync url = async {
        return! Async.Catch(async {
            let! str = Http.AsyncRequestString(url)
            return str })
    }

    let result = fetchAsync "http://www.example.bad"
    match result with      
    | Choice1Of2 v -> logger.LogInformation("worked")
    | Choice2Of2 ex -> logger.LogInformation("failed")
f#-data f#
3 Answers

a) Why it fails

Your fetchAsync function returns an Async<Choice<_>> and you pattern match as if the function only returns Choice<_>. Unfortunately you can't pattern match on Async because that would be a blocking operation, exactly what Async tries move away from.

b) Idiomatic way of dealing with these things.

What you can do, however, is stay within the Async context and handle the failure within. F# provides (at least) two common ways to deal with this. E.g. you can use the async helper functions that allow you to write a pipeline:

let fetchAsyncPipeline (url: string) =
    FSharp.Data.Http.AsyncRequestString(url)
    |> Async.Catch
    |> Async.map (function
        | Choice1Of2 v -> Ok v
        | Choice2Of2 e -> Error e.Message)

Unfortunately, Async.map is not yet included. But you can define it for yourself like this:

namespace global

[<RequireQualifiedAccess>]
module Async =

    let map f xA =
        async {
            let! x = xA
            return f x
        }

Or, you can use F#'s computation expressions that provide syntactic sugar to write the above in a more imperative style:

let fetchAsyncCe (url: string) =
    async {
        try
            return!
                FSharp.Data.Http.AsyncRequestString(url)
                |> Async.map Ok
            with
            | e -> return Error e.Message
    }

In both these solutions, I converted the exception to F#'s result type, which I personally find the nicest way to deal with errors.

Finally, as indicated by brianberns, your expression is only wrapped in the Async type. But unlike C#'s Task, async computations represent the program how to calculate something, but that program has not yet started and you have to explicitly run the asynchronous operation. One of the ways to do this is to use Async.RunSynchronously:

Async.RunSynchronously (fetchAsyncCe "https://fsharpforfunandprofit.com/")

PS: You probably came across Scott Wlaschin's excellent F# for fun and profit, but if you didn't you should check it out. Personally, I've found it the best resource to teach you F# and functional programming in general.

user_22194180

The reason this doesn't compile is because you're never actually running the computation created by fetchAsync. Something like this should work instead:

let fetchAsync url =
    Http.AsyncRequestString(url)
        |> Async.Catch
        |> Async.RunSynchronously
user_3442230

After a few more hours and a lot of coffee I have this

let fetchAsync2 (url: string) =
async { return! Async.Catch(Http.AsyncRequestString(url)) }
|> Async.RunSynchronously

let result = fetchAsync2 url

match result with
| Choice1Of2 html -> html
| Choice2Of2 error -> error.Message

This code works the way I want it to but I'll leave the question open in case anyone has a better solution

Merging my version and the one provided by @brianberns I think I like this one best?

let fetchAsync3 (url: string) =
Http.AsyncRequestString(url)
|> Async.Catch 
|> Async.RunSynchronously

let result = fetchAsync3 url

match result with
| Choice1Of2 html -> html
| Choice2Of2 error -> error.Message
user_17690
Related