Nginx in docker container to execute Perl script in other container. Buffering problem

Viewed 287

I am building the web server where Nginx runs in docker container. And there are another 2 docker containers for PHP code and for Perl code (cgi).

PHP container works very good. But not cgi container to execute Perl. It works slow for file uploads

Nginx config looks like this

........
server {
    listen 443 ssl http2 default_server;
    proxy_buffering                 off;
    proxy_request_buffering         off;
    proxy_http_version              1.1;
    proxy_ignore_client_abort       on;
    client_body_timeout             180s;
    client_max_body_size            0;
    proxy_max_temp_file_size        0;
    proxy_read_timeout              300s;
    proxy_send_timeout              300s;
    proxy_connect_timeout 2s;

    location ~* \.php$ {
         client_body_timeout  180s;
         client_max_body_size 0;
         fastcgi_pass   php:9000;
         include         fastcgi_params;
         fastcgi_param   SCRIPT_FILENAME    $document_root$fastcgi_script_name;
         fastcgi_param   SCRIPT_NAME        $fastcgi_script_name;
         fastcgi_read_timeout 900s;
         fastcgi_send_timeout 900s;
         fastcgi_ignore_client_abort on;
         fastcgi_keep_conn on;
         fastcgi_socket_keepalive on;
    }
    location ~* ^/cgi-bin/.+\.cgi {
         client_body_timeout  172800s;
         client_max_body_size 0;
         gzip off;
         include fastcgi_params;
         fastcgi_pass   cgi:9001;
         fastcgi_request_buffering off;
         fastcgi_read_timeout 900s;
         fastcgi_send_timeout 900s;

         fastcgi_ignore_client_abort on;
         fastcgi_keep_conn on;
         fastcgi_socket_keepalive on;
     }

Inside cgi container i do the following

cat ../nginx-build/cgi/Dockerfile 
FROM ff-base

RUN apt update &&\
    apt upgrade -y &&\
    apt-get install -y spawn-fcgi fcgiwrap

EXPOSE 9001

#CMD ["/usr/sbin/fcgiwrap", "-f", "-s", "tcp:0.0.0.0:9001"]
CMD ["/usr/bin/spawn-fcgi", "-n", "-a", "0.0.0.0","-p","9001","--","/usr/sbin/fcgiwrap"]

My base image has the code to install perl etc. No important here.

So, what is the problem.

When i change the option fastcgi_request_buffering off; to on, all start to work fine with uploads to cgi container. But i need to have it off. I need my perl script reads data immediately users started to post it. I do not want to have buffering. And in general it works. Bu very slowly. Upload with buffering takes 5 sec for 28mb file. But upload of same file without buffering takes 1m20sec.

My perl file is simplest. It only reads input and does nothing with it, only counts. I played with different size of read buffer 1kb,2,4,16 etc. No difference

#!/usr/bin/perl 

logIt("start");

print "Content-type: text/html\n";
print "\n";

logIt("Before read started");

binmode(STDIN);
my $readbuffer;
my $bytes_read=0;

while(!eof(STDIN) && $ENV{"CONTENT_LENGTH"} > 0){
    my $toread = 65636;

        if ($bytes_read + $toread > $ENV{"CONTENT_LENGTH"}) {
            $toread = $ENV{"CONTENT_LENGTH"} - $bytes_read;
        }

        read(STDIN, $readbuffer, $toread);
        logIt("read ".length($readbuffer));     
    $bytes_read+=length($readbuffer);
                
    last if($bytes_read>=$ENV{"CONTENT_LENGTH"});
}
print "Uploaded $bytes_read bytes\n";
logIt("done file read $bytes_read bytes");

sub logIt{
    my $s=shift;

        my $fname = "/logfile.txt";

        if(open LOGFF, ">> $fname"){
            print LOGFF "[".scalar(localtime)."] ".$$.": $s\n";
                close LOGFF;
        }
}

Results of this script shows that data appear in a read buffer regularly but read is slow. Like 5-6 reads per second.

So, my problem is maybe inside /usr/sbin/fcgiwrap or in spawn-fcgi related to how they read data transferred from nginx container (nginx just sends data because no buffering).

Why could this be too slow?

0 Answers
Related