How to get pdf base64 data from httpwebresponse rest api

I am getting a PDF using HttpWebResponse. I have copied the response stream to a memory stream and converted the stream data to base64 using a byte array. I then save the base64 string to a database. When the PDF contains plain text it is working fine. I have tried like

using (HttpWebResponse responsePdf = (HttpWebResponse)request.GetResponse())
{
    Stream stream = responsePdf.GetResponseStream();
    MemoryStream memoryStream = new MemoryStream();
    stream.CopyTo(memoryStream);

    byte[] bytes = memoryStream.ToArray();
    base64PdfData = Convert.ToBase64String(bytes);

    stream.Close();
}

If PDF contains images or highlighted contents, then I don't receive the correct base64 string and the PDF doesn't open.

How can I get the PDF data?

Response Headers

content-type: application/pdf
content-disposition: attachment; filename="test.pdf"

Thanks, Ajay