Puphpeteer - Get text and href-attribute from link

user_2847689 0
Viewed 526

I am using "@nesk/puphpeteer": "^2.0.0" Link to Github-Repo and want get the text and the href-attribute from a link.

I tried the following:

<?php

require_once '../vendor/autoload.php';

use Nesk\Puphpeteer\Puppeteer;
use Nesk\Rialto\Data\JsFunction;

$debug = true;

$puppeteer = new Puppeteer([
    'read_timeout' => 100,
    'debug' => $debug,
]);
$browser = $puppeteer->launch([
    'headless' => !$debug,
    'ignoreHTTPSErrors' => true,
]);

$page = $browser->newPage();
$page->goto('http://example.python-scraping.com/');

//get text and link
$links = $page->querySelectorXPath('//*[@id="results"]/table/tbody/tr/td/div/a', JsFunction::createWithParameters(['node'])
    ->body('return node.textContent;'));

// iterate over links and print each link and its text

// get single text
$singleText = $page->querySelectorXPath('//*[@id="pagination"]/a', JsFunction::createWithParameters(['node'])
    ->body('return node.textContent;'));


$browser->close();

When I run the above script I get the nodes from the page, BUT I cannot access the attributes or the text?

Any suggestions how to do this?

I appreciate your replies!

php
2 Answers

querySelectorXPath return array of ElementHandle. one more thing querySelectorXPath does not support callback function.

first get all node ElementHandle

$links = $page->querySelectorXPath('//*[@id="results"]/table/tbody/tr/td/div/a');

then loop over links to access attributes or text of node

foreach($links as $link){
   // for text
    $text = $link->evaluate(JsFunction::createWithParameters(['node'])
    ->body('return node.innerText;'));

  // for link
  $link = $link->evaluate(JsFunction::createWithParameters(['node'])
    ->body('return node.href;'));
}
user_27765360

Disclaimer: This is just an intermediate answer - I would update once I've got more specific requests on HTML attrs or other expectations to be retrieved.

tl;dr: Mentioned composer package nesk/puphpeteer really is just a wrapper to underlying NodeJS based implementation of puppeteer. Thus, accessing data and structures has to be "similar" to their JavaScript counterparts...

Maybe Codeception (headless) or symfony/dom-crawler (raw markup) might be better and more mature alternatives.

Anyway, let's pick the example from above and go through it step by step:

$links = $page->querySelectorXPath(
  '//*[@id="results"]/table/tbody/tr/td/div/a', 
  JsFunction::createWithParameters(['node'])->body('return node.textContent;')
);

Thus, after that we would have something like this:

$links = $page->querySelectorXPath(
  '//*[@id="results"]/table/tbody/tr/td/div/a', 
  JsFunction::createWithParameters(['node'])->body('return node.textContent;')
);

/** @var \Nesk\Puphpeteer\Resources\ElementHandle $link */
foreach ($links as $link) {
  var_dump($link->getProperty('textContent')->jsonValue());
}

Which outputs the following raw data (as retrieved from http://example.python-scraping.com/):

string(12) " Afghanistan"
string(14) " Aland Islands"
string(8) " Albania"
string(8) " Algeria"
string(15) " American Samoa"
string(8) " Andorra"
string(7) " Angola"
string(9) " Anguilla"
string(11) " Antarctica"
string(20) " Antigua and Barbuda"
user_28541400
Related