PHP script no longer receiving data from context stream

user_8147165 0
Viewed 84

We have a "legacy" script that stopped working a little while back. Pretty sure it's because the endpoint it's connecting to changed from http to https, and the old http address now returns a 301.

I've never done anything other than tiny changes to PHP scripts, so am a little out of my depth here.

Note that our PHP version is old - 5.3.0. This may well be part of the problem.

The script as-is (relevant bit anyway):

$uri = "http://www.imf.org/external/np/fin/data/rms_mth.aspx"
     ."?SelectDate=$date&reportType=CVSDR&tsvflag=Y";
$opts = array('http' => array(
                 'proxy' => 'tcp://internal.proxy.address:port',
                 'method' => 'GET',
                 'request_fulluri' => true)
              );
$ctx = stream_context_create($opts);
$lines = file($uri, false, $ctx);
foreach ($lines as $line)
...

This returns nothing any more. The link btw is the IMF link for exchange rates, so that is open to all - if you open it you'll get a download with a rate table in it. The rest of the script basically parses this for the data we want.

Now, pretty sure our proxy is OK. Running some tests with curl gives the following results:

curl --proxy tcp://internal.proxy.address:port -v https://www.imf.org/external/np/fin/data/rms_mth.aspx?SelectDate=05/28/2020&reportType=CVSDR&tsvflag=Y

(specify https) works just fine.

curl --proxy tcp://internal.proxy.address:port -v http://www.imf.org/external/np/fin/data/rms_mth.aspx?SelectDate=05/28/2020&reportType=CVSDR&tsvflag=Y

(specify http) does not work, and shows a 301 error

curl --proxy tcp://internal.proxy.address:port -v -L http://www.imf.org/external/np/fin/data/rms_mth.aspx?SelectDate=05/28/2020&reportType=CVSDR&tsvflag=Y

(specify http with follow redirects) then works OK.

I've tried a few things after some googling. It seems I need opts for 'ssl' as well when using https. So I've made the following changes

$uri = "https://www.imf.org/external/np/fin/data/rms_mth.aspx"
     ."?SelectDate=$date&reportType=CVSDR&tsvflag=Y";
$opts = array('http' => array(
                 'proxy' => 'tcp://internal.proxy.address:port',
                 'method' => 'GET',
                 'request_fulluri' => true),
              'ssl' => array(
                 'verify_peer' => false,
                 'verify_peer_name' => false,
                 'SNI_enabled' => false)
              );

Sadly, the SNI_enabled flag was introduced after 5.3.0, so I don't think this helps. There's also a follow_location context option for http, but that was introduced in 5.3.4, so also no use.

(BTW, I have little to no control over the version of PHP we have, so while I appreciate higher versions may offer better solutions, that's not a lot of use to me I'm afraid).

Basically, I am now stuck. No combination of these parameters or settings returns any data at all. I can see it works via curl and the proxy, so it's not a general connectivity issue.

Any and all suggestions gratefully received!

Update: After adding the lines to enable error reporting, the error code is for the stream connecting: 

Warning: file(https://www.imf.org/external/np/fin/data/rms_mth.aspx?SelectDate=05/28/2020&reportType=CVSDR&tsvflag=Y): failed to open stream: Cannot connect to HTTPS server through proxy in /usr/bass/apps/htdocs/BASS/mods/module.XSM.php on line 79

(line 79 is the $lines = ... line)

So it doesn't connect in the php script, but running the same connection via the proxy in curl works fine. What's the difference in php that causes this?

php
1 Answers

You can use php curl functions to get the response from your given url. And then you can use explode() function to break the response line by line.

$uri = "https://www.imf.org/external/np/fin/data/rms_mth.aspx"
     ."?SelectDate=$date&reportType=CVSDR&tsvflag=Y";

$opts = array(
    CURLOPT_URL            => $uri,
    CURLOPT_PROXY          => 'tcp://internal.proxy.address:port',
    CURLOPT_HEADER         => false,
    CURLOPT_RETURNTRANSFER => true
);

$ch = curl_init();
curl_setopt_array($ch, $opts);
$lines = curl_exec($ch);
curl_close($ch);

$lines = explode("\n", $lines);    // breaking the whole response string line by line

foreach ($lines as $line)
...
user_56742230
Related