Piping a stream to a warp response

Viewed 468

I would like to create a warp endpoint that would stream a specific requested file (from a Google bucket in my case). I don't want to load the whole file in memory before sending it back to the user because it could be rather big.

I'm using the cloud_storage crate to get the file :

pub async fn download_streamed(
    bucket: &str,
    file_name: &str
) -> Result<impl Stream<Item = Result<u8>> + Unpin>

(see doc here)

Here's the code that is bothering me:

pub async fn download(file_name: String) -> Result<impl Reply, Infallible> {
    let stream = Client::default()
        .object()
        .download_streamed(
            &var("BUCKET").expect("Missing `BUCKET` env var"),
            &file_name,
        )
        .await
        .unwrap();

    let body = hyper::Body::wrap_stream(stream);
                                        ^^^^^^

    Ok(warp::reply::Response::new(body))
}

This is the compiler's message:

112 |     let body = hyper::Body::wrap_stream(stream);
    |                ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `From<u8>` is not implemented for `hyper::body::Bytes`
    |
   ::: /Users/.../.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.14.12/src/body/body.rs:193:12
    |
193 |         O: Into<Bytes> + 'static,
    |            ----------- required by this bound in `Body::wrap_stream`
    |
    = help: the following implementations were found:
              <hyper::body::Bytes as From<&'static [u8]>>
              <hyper::body::Bytes as From<&'static str>>
              <hyper::body::Bytes as From<Box<[u8]>>>
              <hyper::body::Bytes as From<Vec<u8>>>
            and 5 others
    = note: required because of the requirements on the impl of `Into<hyper::body::Bytes>` for `u8`

As I understand it - and because the message is pretty clear too - the stream's associated Item is of type u8 and there is no From<u8> for hyper::body::Bytes available.

Would there be a way to do some conversion magic then?

--

Edit: I tried a few things :

  • First, I naively tried to map_ok the u8 to a vec![u8]. It worked but the performances were beyond poor

  • Secondly, I modified download_streamed implementation to return a stream of Bytes :

    pub async fn download_streamed(
        &self,
        bucket: &str,
        file_name: &str,
    ) -> crate::Result<impl Stream<Item = std::result::Result<Bytes, reqwest::Error>> + Unpin> {

        [... omitted ...]

        let response = self
            .0
            .client
            .get(&url)
            .headers(self.0.get_headers().await?)
            .send()
            .await?
            .error_for_status()?;

        Ok(response.bytes_stream())

Result: blazing fast ...

0 Answers
Related