How to get file from POST data in Bash CGI script?

user_275221 0
Viewed 11160

I am trying to post a file using cURL and receive it on the other side via a CGI Bash script and store it with the same name. After upload is completed, diff between the original file and reconstructed one should return zero.

The way cURL sends data:

curl --request POST --data-binary "@dummy.dat" 127.0.0.1/cgi-bin/upload-rpm

Receiver script:

#!/bin/bash

echo "Content-type: text/html"
echo ""

echo '<html>'
echo '<head>'
echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'
echo '<title>Foo</title>'
echo '</head>'
echo '<body>'

echo "<p>Start</p>"

if [ "$REQUEST_METHOD" = "POST" ]; then
    echo "<p>Post Method</p>"
    if [ "$CONTENT_LENGTH" -gt 0 ]; then
        echo "<p>Size=$CONTENT_LENGTH</p>"
        while read -n 1 byte -t 3; do echo -n -e "$byte" >> ./foo.dat ; done
    fi
fi
echo '</body>'
echo '</html>'

exit 0

But it's not working. File is not created on the server side. And how can I get the file name?

cgi curl bash
2 Answers

as long as you're always uploading exactly 1 file using the multipart/form-data format and the client always put the Content-Type as the last header of the file upload (which curl always seem to do), this seem to work:

#!/bin/bash

echo "Content-type: text/html"
echo ""

echo '<html>'
echo '<head>'
echo '<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">'
echo '<title>Foo</title>'
echo '</head>'
echo '<body>'

echo "<p>Start</p>"

if [ "$REQUEST_METHOD" = "POST" ]; then
    echo "<p>Post Method</p>"
    if [ "$CONTENT_LENGTH" -gt 0 ]; then
    in_raw="$(cat)"
    boundary=$(echo -n "$in_raw" | head -1 | tr -d '\r\n');
    filename=$(echo -n "$in_raw" | grep --text --max-count=1 -oP "(?<=filename=\")[^\"]*");
    file_content=$(echo -n "$in_raw" | sed '1,/Content-Type:/d' | tail -c +3 | head --lines=-1 | head --bytes=-4  );
    echo "boundary: $boundary"
    echo "filename: $filename"
    #echo "file_content: $file_content"
    fi
fi
echo '</body>'
echo '</html>'

exit 0

example upload invocation (which I used to debug the above):

curl http://localhost:81/cgi-bin/wtf.sh -F "MyFile=@myfile.txt"
  • with all that said, THIS IS INSANITY! bash is absolutely not suitable for handling binary data, such as http file uploads, use something else. (PHP? Python? Perl?)
user_10670030

The ways curl sends data

With fields:

curl --data "param1=value1&param2=value2" https://example.com/resource.cgi

With fields specified individually:

curl --data "param1=value1" --data "param2=value2" https://example.com/resource.cgi

Multipart:

curl --form "fileupload=@my-file.txt" https://example.com/resource.cgi

Multipart with fields and a filename:

curl --form "fileupload=@my-file.txt;filename=desired-filename.txt" --form param1=value1 --form param2=value2 https://example.com/resource.cgi

For a RESTful HTTP POST containing XML:

curl -X POST -d @filename.txt http://example.com/path/to/resource --header "Content-Type:text/xml"

or for JSON, use this:

curl -X POST -d @filename.txt http://example.com/path/to/resource --header "Content-Type:application/json"

This will read the contents of the file named filename.txt and send it as the post request.

For further information see

Reading HTTP POST data using BASH

How to get the content length:

if [ "$REQUEST_METHOD" = "POST" ]; then
    if [ "$CONTENT_LENGTH" -gt 0 ]; then
        read -n $CONTENT_LENGTH POST_DATA <&0
        echo "$CONTENT_LENGTH"
    fi
fi

To save a binary or text file:

boundary=$(export | \
    sed '/CONTENT_TYPE/!d;s/^.*dary=//;s/.$//')
filename=$(echo "$QUERY_STRING" | \
    sed -n '2!d;s/\(.*filename=\"\)\(.*\)\".*$/\2/;p')
file=$(echo "$QUERY_STRING" | \
    sed -n "1,/$boundary/p" | sed '1,4d;$d')

The uploaded file is now contained in the $file variable, and file name in the $filename variable.

user_98018300
Related