Why is Node.js htmlparser2 inserting additional nodes while parsing a document?

user_10838727 0
Viewed 200

I'm trying to fetch html content of a webpage using node and then process it's whole DOM to construct something(irrelevant) corresponding to that html. So it's very important to get the exact DOM representation from that html and then be able to process it. I'm using htmlparser2 for that purpose.

But apparently, the parser is sometimes not producing the exact DOM representation and inserting some additional tags on it's own. Here is a simple example:

const htmlparser2 = require("htmlparser2");

const handler = new htmlparser2.DomHandler(function(err, dom) {
  if (err) console.error(err);
  else console.log(dom);
});

const parser = new htmlparser2.Parser(handler);
parser.write("<center>center</center><p>paragraph1 <p>paragraph2</p></p>");
parser.end();

Obviously the output should have two root nodes(one for the center tag and one for the p tag). But the console outputs four root nodes! And the structure seems weird too.

The expected DOM structure:

- center
  - text
- p
  - text
  - p
    - text

Versus the output structure:

- center
  - text
- p
  - text
- p
  - text
- p

My Stackoverflow formatted one liner question is: Why is the parsed output structure of htmlparser2 different from the source DOM structure?

Followup questions: Is this the expected behavior of htmlparser2? If it is, then I'd greatly appreciate any suggestions for an alternative.

Thank you very much in advance.

node.js web-scraping javascript
1 Answers

You cannot nest a <p> element inside another <p> element like that. <p> is a block-level element. <p> elements are automatically closed if another block level element is found before the </p>. The parser is correct. My only suggestion would be to fix your HTML.

https://developer.mozilla.org/en-US/docs/Web/HTML/Element/p

Just to further illustrate, I put your HTML into the snippet tool here on StackOverflow and inspected the HTML in Dev Tools and this is what I got:

<center>center</center>
<p>paragraph1 </p>
<p>paragraph2</p>
<p></p>

So the parser is giving you exactly the same DOM as what Firefox gives me for that same HTML.

mlibby_134680
Related