Sending chunked data a grpc ServerStream

I have this RPC defined

rpc Download(DownloadRequest) returns (stream google.api.HttpBody)

I'd eventually like to use this to serve both CSV and binary data, but I'm starting with CSV.

In my handler I am doing the following to repeatedly grab the data from storage and send it to the stream:

if err := stream.SetHeader(metadata.Pairs("content-disposition", "attachment")); err != nil {
    return err
}
chunkSize := uint64(100) // arbitrary chunk size
for offset := uint64(0); offset < objLen; offset += chunkSize {
    contents, err := getContent(offset, chunkSize)
    if err != nil {
        return err
    }
    err = stream.Send(&httpbody.HttpBody{
        ContentType: "text/csv",
        Data:        contents,
    })
    if err != nil {
        return err
    }
}

The problem with this implementation is that the CSV result ends up with line breaks every chunkSize characters. How can I avoid the newline after every chunk in my output?

I am using grpc-gateway to serve this via REST, in case that matters.