How to download file from URL with a lot of redirects

user_13423729 0
Viewed 266

I'm trying to get content of XML file from URL to Google Sheets using Apps Script (JS)

Unfortunately URL has lots of strange redirects and I have no impact on this on server side. I need to handle it by my side in JS. In Firefox I was able to download file normally but in JS it's a challenge.

I was searched for solution for days and nothing helps. JS just can't get file content.

URL: https://www.nife.pl/getfile-woocommerce/b55a488229b69d7c4ddf2a9f1a6b5dcb.xml

I've tried those:

  • Default importXML in Goolge Sheets cell like this:

=importXml("https://www.nife.pl/getfile-woocommerce/b55a488229b69d7c4ddf2a9f1a6b5dcb.xml"; "//variant")

  • UrlFetchApp:
function test(){
  var url = 'https://www.nife.pl/getfile-woocommerce/b55a488229b69d7c4ddf2a9f1a6b5dcb.xml';
  var xml = UrlFetchApp.fetch(url,{'followRedirects': true, 'muteHttpExceptions': true}).getContentText();
  var jsonV = XML_to_JSON(xml);
  Logger.log(jsonV);
}
  • XMLService:

var xml = XmlService.parse('https://www.nife.pl/getfile-woocommerce/b55a488229b69d7c4ddf2a9f1a6b5dcb.xml',{'followRedirects': true, 'muteHttpExceptions': true});

  • tried followRedirects && muteHttpExceptions true and also false

Nothing helps.

Any idea how to get content from this XML to GoogleSheets?

google-sheets-formula google-apps-script google-sheets xml javascript
2 Answers

The following code works:

function getXml() {
  var response = UrlFetchApp.fetch(
    'http://www.nife.pl/getfile-woocommerce/b55a488229b69d7c4ddf2a9f1a6b5dcb.xml?waluta=EUR',
    {
      headers: {Cookie: 'ip2loc=isset'}, 
    }
  );
  if (response.getResponseCode() == 200) {
    var content = response.getContentText();
    Logger.log(content.length);  // 1612024.0 in my test
  }
}

Please note, the above URL was taken from the first response "Location" header, when HTTP response code was equal to 302. The same response showed one more header: "Set-cookie:ip2loc=isset". I quess this cookie is important to receive full data, so we should implement it in request as a header. Really, after having corrected URL and added cookie header we obtain XML as any browser does.

user_108051620

By analyzing the request sent by the browser, it seems, that you have to add the following value to the request header

{'cookie': 'ip2loc=isset'}

So your function would look like this:

function myFunction() {
  var url = 'http://www.nife.pl/getfile-woocommerce/b55a488229b69d7c4ddf2a9f1a6b5dcb.xml';
  
  var response = UrlFetchApp.fetch(url, { headers : {'cookie': 'ip2loc=isset'}} ).getContentText();
  Logger.log(response);
}
user_13907910
Related