HttpClient.GetByteArrayAsync giving 500 error - how can I get more detailed message?

I have a HttpClient making a request for a file (from sharepoint, though I don't know if that's relevant). It's working well when called from one area of the application but fails when called from a different area, and I'm trying to debug why this is. However, I get the following error which is quite vague:

System.Net.Http.HttpRequestException: Response status code does not indicate success: 500 (Internal Server Error).

This is thrown by client.GetByteArrayAsync (code below) - is it possible to get a more detailed error message from this?

private async Task<byte[]> RequestBlobData(string url)
{
    var myCache = GetCredentialCache(url);

    using (var client = new HttpClient(new HttpClientHandler { Credentials = myCache }))
    {
        client.DefaultRequestHeaders.Add("Accept", "application/octet-stream");
        client.DefaultRequestHeaders.Add("binaryStringResponseBody", "true");

        return await client.GetByteArrayAsync(new Uri(url));
    }
}

(Just to be clear, I'm not expecting someone to be able to fix the actual error, I'm hoping that someone will know how I can get a more detailed error message.)