How to scrape URL protected by login using Goutte (I have login account)

Viewed 1366

I found similar question in here. But I didn't get enough information, so I decided to make new question.

let's assume urls are as following.

url1. http://base_url/login
url2. http://base_url/home
url3. http://base_url/target

Note: if I logged in url1, site redirects to url2 after login and
      2 cookies are saved on browser(called CTID, CTP).

I want to scrap url3 but url3 can be accessed after login.

I wrote program like this.

use Goutte\Client;
... ...
$client = new Client();
$client->setHeader('User-Agent', "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.101 Safari/537.36");

$crawler = $client->request('GET', 'url1');
$form = $crawler->selectButton('LOGON')->form();
$crawler = $client->submit($form, array('ID' => '***', 'PASS' => '***'));
dump($crawler->html());  //1

$crawler = $client->request('GET', 'url3');
dump($crawler->html());   //2

Instruction1 outputs url2's result correctly.(that means login is successful.) I but I couldn't get url3's content. How can I do for getting url3's content? Thank you for reading my question.

0 Answers
Related