Stream entire file via conduit

Question

I am struggling to download a (dynamic) file via Servant.

Say I have the following function for pulling files from S3, using this library: https://hackage.haskell.org/package/aws:

getS3Object
  :: MonadIO m
  => MonadReader Config m
  => MonadError CustomError m
  => Text
  -> m (Maybe ByteString)

getS3Object fileName = do
    config <- ask
    configObject <- constructS3Authentication
    mgr <- liftIO $ newManager tlsManagerSettings
    liftIO $ runResourceT $
      Aws.pureAws (awsConfig configObject) (s3Config configObject) mgr (S3.getObject (config ^. awsCustomerDocumentsBucket) fileName) >>= 
        \gor -> runConduit (responseBody (S3.gorResponse gor) .| await)

I then call this like so:

fetchDocument
  :: MonadIO m
  => MonadCatch m
  => MonadReader Config m
  => MonadError CustomError m
  => NewDocumentInput
  -> m ByteString

fetchDocument inputDocument = do
    let fileName = computeFileName inputDocument
    documentObject <- getS3Object fileName
    case documentObject of
      Nothing -> throwError $ OtherAWSError "No document returned, or other uncaught exception"
      Just document -> liftIO $ do    
        writeFile "./tmp/potato.pdf" document
        return document

When I call this functionality (by calling the requisite route via Postman), Potato.pdf is a 4kb file which seems like it might be a part of my PDF file but not all of it. This file can not be opened.

Oddly enough, when I change the last line of getS3Object to:

\gor -> runConduit (responseBody (S3.gorResponse gor) .| sinkFile "./tmp/foo.pdf" .| await)

the file that gets output to disk is the full correct (33KB) PDF file (whichc can be opened).

My question is, where am I losing the remainder of the file? Am I using Conduit incorrectly somehow? And how can I fix this issue.

Stream entire file via conduit

0 Answers