curl error 18 - transfer closed with outstanding read data remaining

user_173036 0
Viewed 202479

when retrieving data from a URL using curl, I sometimes (in 80% of the cases) get

error 18: transfer closed with outstanding read data remaining

Part of the returned data is then missing. The weird thing is that this does never occur when the CURLOPT_RETURNTRANSFER is set to false, that is the curl_exec function doesn't return the data but displays the content directly.

What could be the problem? Can I set some of the options to avoid such behaviour?

curl php
14 Answers

Seeing this error during the use of Guzzle as well. The following header fixed it for me:

'headers' => [
    'accept-encoding' => 'gzip, deflate',
],

I issued the request with Postman which gave me a complete response and no error. Then I started adding the headers that Postman sends to the Guzzle request and this was the one that fixed it.

user_42832440

Encountered similar issue, my server is behind nginx. There's no error in web server's (Python flask) log, but some error messsage in nginx log.

[crit] 31054#31054: *269464 open() "/var/cache/nginx/proxy_temp/3/45/0000000453" failed (13: Permission denied) while reading upstream

I fixed this issue by correcting the permission of directory:

/var/cache/nginx
user_106779240

I got this error when my server ran out of disk space and closed the connection midway during generating the response and simply closed the connection

user_137756860

I got this error when i was accidentally downloading a file onto itself.
(I had created a symlink in an sshfs mount of the remote directory to make it available for download, forgot to switch the working directory, and used -OJ).

I guess it won’t really »help« you when you read this, since it means your file got trashed.

user_27479620

I had this same problem. I tried all of these solutions but none worked. In my case, the request was working fine in Postman but when I do it with curl in php I get the error mentioned above.

What I did was check the PHP code generated by Postman and replicate the same thing.

First the request is set to use Http version 1.1 And the second most important part is the encoding for me.

Here is the code that helped me

curl_setopt($ch, CURLOPT_ENCODING, '');
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);

If I remove the CurlOpt Encoding I get back the error.

user_103417680

I got this error when running through a nginx proxy and I was running nginx under the user-id daemon instead of the user id nginx.

This means some of nginx's scratch directories weren't accessible / writable.

Switching from user daemon; to user nginx; fixed it for me.

user_41024060

it can be related to many issues. In my case, i was using Curl to build an image (via Docker api). Thus, the build was stuck that's why i got this error. when I fixed the build, the error disappeared.

user_132470340

We can fix this by suppressing the Expect: 100-continue header that cURL normally sends.

user_172866950
Related