How to download internet file locally in Haskell

Viewed 234

I'm struggling with a simple task of downloading a file locally. Can someone point me to a library and a sample code that download say a file locally ?

Thanks!

1 Answers

I think the most common packages for this purpose are http-client, http-conduit, wreq, and perhaps req. All appear fairly well maintained. The last three are actually based on http-client and just try to provide a slightly friendlier interface (http-conduit and wreq) or a more type-safe interface (req). For the simple task of downloading a file, there's not much to choose between the first three. The fourth (req) is maybe overkill because it's somewhat more complicated to use, though I've included an example at the end.

To use http-client for HTTPS connections, you need to install both the http-client and http-client-tls packages. Sample code to retrieve and save a downloaded file would look like this:

import Network.HTTP.Client          -- package http-client
import Network.HTTP.Client.TLS      -- package http-client-tls
import qualified Data.ByteString.Lazy as BL

main = do
  -- create a connection manager
  manager <- newManager tlsManagerSettings
  -- create the request
  request <- parseRequest "https://www.google.com/robots.txt"
  -- make the request
  r <- httpLbs request manager
  -- get the contents (as a lazy ByteString)
  let contents = responseBody r
  -- write it to a local file
  BL.writeFile "myfile.txt" contents

For http-conduit, the code is similar. It just drops the requirement for using an explicit "manager" and allows you to construct a request directly from a text string using the OverloadedStrings extension:

{-# LANGUAGE OverloadedStrings #-}

import Network.HTTP.Simple           -- package http-conduit
import qualified Data.ByteString.Lazy as BL

main = do
  -- make the requeset
  r <- httpLBS "https://www.google.com/robots.txt"
  -- get the contents (as a lazy ByteString)
  let contents = getResponseBody r
  -- write it to a local file
  BL.writeFile "myfile.txt" contents

The wreq package uses the lens package and has a pretty clean interface:

import Network.Wreq                  -- package wreq
import Control.Lens                  -- package lens
import qualified Data.ByteString.Lazy as BL

main = do
  -- make a requeset
  r <- get "https://www.google.com/robots.txt"
  -- get the contents (as a lazy ByteString)
  let contents = r ^. responseBody
  -- write it to a local file
  BL.writeFile "myfile.txt" contents

The req package has a different design philosophy which makes its main request call look a fair bit more complicated:

{-# LANGUAGE OverloadedStrings #-}

import Network.HTTP.Req               -- package req
import qualified Data.ByteString.Lazy as BL

main = do
  -- make a requeset
  r <- runReq defaultHttpConfig $
    req GET (https "www.google.com" /: "robots.txt")
        NoReqBody lbsResponse mempty
  -- get the contents (as a lazy ByteString)
  let contents = responseBody r
  -- write it to a local file
  BL.writeFile "myfile.txt" contents
Related