How do you parse and process HTML/XML in PHP?

user_353790 0
Viewed 460749

How can one parse HTML/XML and extract information from it?

xml-parsing xml html-parsing html php
31 Answers

Just use DOMDocument->loadHTML() and be done with it. libxml's HTML parsing algorithm is quite good and fast, and contrary to popular belief, does not choke on malformed HTML.

Ambush_Commander_238450

Simple HTML DOM is a great open-source parser:

simplehtmldom.sourceforge

It treats DOM elements in an object-oriented way, and the new iteration has a lot of coverage for non-compliant code. There are also some great functions like you'd see in JavaScript, such as the "find" function, which will return all instances of elements of that tag name.

I've used this in a number of tools, testing it on many different types of web pages, and I think it works great.

Robert_Elwell_231020

QueryPath is good, but be careful of "tracking state" cause if you didn't realise what it means, it can mean you waste a lot of debugging time trying to find out what happened and why the code doesn't work.

What it means is that each call on the result set modifies the result set in the object, it's not chainable like in jquery where each link is a new set, you have a single set which is the results from your query and each function call modifies that single set.

in order to get jquery-like behaviour, you need to branch before you do a filter/modify like operation, that means it'll mirror what happens in jquery much more closely.

$results = qp("div p");
$forename = $results->find("input[name='forename']");

$results now contains the result set for input[name='forename'] NOT the original query "div p" this tripped me up a lot, what I found was that QueryPath tracks the filters and finds and everything which modifies your results and stores them in the object. you need to do this instead

$forename = $results->branch()->find("input[name='forname']")

then $results won't be modified and you can reuse the result set again and again, perhaps somebody with much more knowledge can clear this up a bit, but it's basically like this from what I've found.

user_2791470

You could try using something like HTML Tidy to cleanup any "broken" HTML and convert the HTML to XHTML, which you can then parse with a XML parser.

CesarB_282580

XML_HTMLSax is rather stable - even if it's not maintained any more. Another option could be to pipe you HTML through Html Tidy and then parse it with standard XML tools.

user_181800

There are several reasons to not parse HTML by regular expression. But, if you have total control of what HTML will be generated, then you can do with simple regular expression.

Above it's a function that parses HTML by regular expression. Note that this function is very sensitive and demands that the HTML obey certain rules, but it works very well in many scenarios. If you want a simple parser, and don't want to install libraries, give this a shot:

function array_combine_($keys, $values) {
    $result = array();
    foreach ($keys as $i => $k) {
        $result[$k][] = $values[$i];
    }
    array_walk($result, create_function('&$v', '$v = (count($v) == 1)? array_pop($v): $v;'));

    return $result;
}

function extract_data($str) {
    return (is_array($str))
        ? array_map('extract_data', $str)
        : ((!preg_match_all('#<([A-Za-z0-9_]*)[^>]*>(.*?)</\1>#s', $str, $matches))
            ? $str
            : array_map(('extract_data'), array_combine_($matches[1], $matches[2])));
}

print_r(extract_data(file_get_contents("http://www.google.com/")));
user_17729900

The best method for parse xml:

$xml='http://www.example.com/rss.xml';
$rss = simplexml_load_string($xml);
$i = 0;
foreach ($rss->channel->item as $feedItem) {
  $i++;
  echo $title=$feedItem->title;
  echo '<br>';
  echo $link=$feedItem->link;
  echo '<br>';
  if($feedItem->description !='') {
    $des=$feedItem->description;
  } else {
    $des='';
  }
  echo $des;
  echo '<br>';
  if($i>5) break;
}
Admin1

If you're familiar with jQuery selector, you can use ScarletsQuery for PHP

<pre><?php
include "ScarletsQuery.php";

// Load the HTML content and parse it
$html = file_get_contents('https://www.lipsum.com');
$dom = Scarlets\Library\MarkupLanguage::parseText($html);

// Select meta tag on the HTML header
$description = $dom->selector('head meta[name="description"]')[0];

// Get 'content' attribute value from meta tag
print_r($description->attr('content'));

$description = $dom->selector('#Content p');

// Get element array
print_r($description->view);

This library usually taking less than 1 second to process offline html.
It also accept invalid HTML or missing quote on tag attributes.

user_65632000

There are many ways:

In General:

  • Native XML Extensions: they come bundled with PHP, are usually faster than all the 3rd party libs, and give me all the control you need over the markup.

  • DOM: DOM is capable of parsing and modifying real-world (broken) HTML and it can do XPath queries. It is based on libxml.

  • XML Reader: XMLReader, like DOM, is based on libxml. The XMLReader extension is an XML pull parser. The reader acts as a cursor going forward on the document stream and stopping at each node on the way

  • XML Parser: This extension lets you create XML parsers and then define handlers for different XML events. Each XML parser also has a few parameters you can adjust. It implements a SAX style XML push parser.

  • Simple XML: The SimpleXML extension provides a very simple and easily usable toolset to convert XML to an object that can be processed with normal property selectors and array iterators.

3rd Party Libraries [ libxml based ]:

  • FluentDom - Repo: FluentDOM provides a jQuery-like fluent XML interface for the DOMDocument in PHP. It can load formats like JSON, CSV, JsonML, RabbitFish and others. Can be installed via Composer.

  • HtmlPageDom: is a PHP library for easy manipulation of HTML documents using It requires DomCrawler from Symfony2 components for traversing the DOM tree and extends it by adding methods for manipulating the DOM tree of HTML documents.

  • ZendDOM: Zend_Dom provides tools for working with DOM documents and structures. Currently, they offer Zend_Dom_Query, which provides a unified interface for querying DOM documents utilizing both XPath and CSS selectors.

  • QueryPath: QueryPath is a PHP library for manipulating XML and HTML. It is designed to work not only with local files but also with web services and database resources. It implements much of the jQuery interface (including CSS-style selectors), but it is heavily tuned for server-side use. Can be installed via Composer.

  • fDOM Document: fDOMDocument extends the standard DOM to use exceptions at all occasions of errors instead of PHP warnings or notices. They also add various custom methods and shortcuts for convenience and to simplify the usage of DOM.

  • Sabre/XML: sabre/xml is a library that wraps and extends the XMLReader and XMLWriter classes to create a simple "xml to object/array" mapping system and design pattern. Writing and reading XML is single-pass and can therefore be fast and require low memory on large xml files.

  • FluidXML: FluidXML is a PHP library for manipulating XML with a concise and fluent API. It leverages XPath and the fluent programming pattern to be fun and effective.

3rd Party Libraries [ Not libxml based ]:

  • PHP Simple HTML DOM Parser: An HTML DOM parser written in PHP5+ lets you manipulate HTML in a very easy way, It Requires PHP 5+. Also Supports invalid HTML. It Extracts contents from HTML in a single line. The codebase is horrible and very slow in working.

  • PHP Html Parser: HPHtmlParser is a simple, flexible, HTML parser that allows you to select tags using any CSS selector, like jQuery. The goal is to assist in the development of tools that require a quick, easy way to scrape HTML, whether it's valid or not. It is slow and takes too much CPU Power.

  • Ganon (recommended): A universal tokenizer and HTML/XML/RSS DOM Parser. It has the Ability to manipulate elements and their attributes. It Supports invalid HTML and UTF8. It Can perform advanced CSS3-like queries on elements (like jQuery -- namespaces supported). A HTML beautifier (like HTML Tidy). Minify CSS and Javascript. It Sort attributes, change character case, correct indentation, etc. Extensible. The Operations separated into smaller functions for easy overriding and Fast and Easy to use.

Web Services:

  • If you don't feel like programming PHP, you can also use Web services. ScraperWiki's external interface allows you to extract data in the form you want for use on the web or in your own applications. You can also extract information about the state of any scraper.

I have shared all the resources, you can choose according to your taste, usefulness, etc.

user_105894260
Related