php: cannot perform multiple fread() calls on php://input

Viewed 690

I'm sending data back and forth to a PHP application using content-encoding: chunked through POSTs. I need my PHP application to read some data, work on it, send back a response, read some more data, and so on. I cannot read all data at once as it won't be available. Imagine a large file upload with a checksum being sent as a response at regular intervals.

The problem is that while I can read a handful of bytes from php://input, subsequent calls to fread do not return the new content.

At the moment I'm using PHP's Docker container. I tried both php:7.0-apache and php:5-apache with the same result.

The PoC client below generates random strings and sends them as chunks to the server at 3-second intervals. The server reads from php://input at 1-second intervals and prints the content. The server output shows only the first three strings are read; also the server seems to 'block' until the first three are read.

Things I've tried, to no avail:

  • Using fseek
  • Using stream_select does not seem to work with, er, the php://input stream. I have no idea why as this would be ideal for me, but given how poorly PHP is designed and implemented I'm not surprised.
  • Closing and re-opening php://input
  • Using fgetc

Client output:

    $ python poc.py
Sending:
---
POST /poc.php HTTP/1.1
Host: localhost
accept-encoding: *;q=0
Transfer-Encoding: chunked
Content-Type: application/octet-stream


---

After sending headers, response:
 HTTP/1.1 200 OK
Date: Mon, 29 May 2017 14:25:52 GMT
Server: Apache/2.4.10 (Debian)
X-Powered-By: PHP/5.6.30
transfer-encoding: chunked
Content-Type: application/octet-stream

4
OK


Waiting 3 seconds
Sending string: AuVuvsyGJc

Waiting 3 seconds
Sending string: LfKouYzccV

Waiting 3 seconds
Sending string: WmpPspYqiR

Waiting 3 seconds
Sending string: IApMOjoaIv

Waiting 3 seconds
Sending string: tuGrVklcVy

Waiting 3 seconds
Sending string: btUVIezCow

Waiting 3 seconds
Sending string: XUPOrEidyd

Traceback (most recent call last):
  File "poc.py", line 33, in <module>
    websock.send(to_chunk(rnd))
socket.error: [Errno 32] Broken pipe

Server output:

Connected
Read: AuVuvsyGJc
LfKouYzccV
WmpPspYqiR

Read:
Read:
Read:
Read:
172.17.0.1 - - [29/May/2017:14:25:52 +0000] "POST /poc.php HTTP/1.1" 200 191 "-" "-"

PHP server:

<?php
header("transfer-encoding: chunked");
header("content-type: application/octet-stream");
flush(); 
/**
 * Useful to print debug messages in the Apache logs
 */
function _log($what) {
    file_put_contents("php://stderr", print_r($what, true) . "\n");
}
_log("Connected");

/**
 * To send data as chunks
 */
function _ch($chunk) {
    echo sprintf("%x\r\n", strlen($chunk));
    echo $chunk;
    echo "\r\n";
    flush();
}
// Test chunks
_ch("OK\r\n");

$web_php_input = fopen("php://input", 'r');
$continue = 5;
while ($continue-- > 0) {
    $contents = fread($web_php_input, 1024);
    _log("Read: " . $contents);
    sleep(1);
}
fclose($web_php_input);
?>

Python client:

from __future__ import print_function
import random
import socket
import string
import time

def to_chunk(what):
    return format(len(what), 'X') + "\r\n" + what + "\r\n"

websock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
websock.connect(("localhost", 8080))

# Send the initial chunked POST header
connect_string = ''.join((
    "POST /poc.php HTTP/1.1\r\n",
    "Host: localhost\r\n",
    "accept-encoding: *;q=0\r\n",  # ,gzip;q=0,deflate;q=0\r\n",
    "Transfer-Encoding: chunked\r\n",
    "Content-Type: application/octet-stream\r\n",
    # "Connection: keep-alive\r\n",
    "\r\n",
))
print("Sending:\n---\n{}\n---\n".format(connect_string))
websock.sendall(connect_string)
print("After sending headers, response:\n {}".format(websock.recv(1024)))
c = True
while c:
    print("Waiting 3 seconds")
    time.sleep(3)
    rnd = ''.join(random.choice(string.ascii_letters) for _ in range(10))
    rnd += '\r\n'
    print("Sending string: {}".format(rnd))
    websock.send(to_chunk(rnd))
print("done")

Dockerfile:

FROM php:5-apache
COPY custom.ini /usr/local/etc/php/conf.d

Docker command line:

docker build -t listener .
docker run -i --rm -p 8080:80 -v $(pwd):/var/www/html --name listener listener

custom.ini file to let PHP know that POST body should not be buffered:

enable_post_data_reading=false

Before someone else suggests using another language, or framework, or doing things differently: it has to be PHP; it cannot rely on any third-party library or PECL; and this is precisely what I need.

As a side note, this behaviour is compliant with the HTTP spec; a server does not have to read all inbound data before returning part of a response to a client. See also RFC6202.

1 Answers
Related