I need to match all of these opening tags:
<p>
<a href="foo">
But not these:
<br />
<hr class="foo" />
I came up with this and wanted to make sure I've got it right. I am only capturing the a-z.
<([a-z]+) *[^/]*?>
I believe it says:
/, thenDo I have that right? And more importantly, what do you think?
In shell, you can parse HTML using sed:
Related (why you shouldn't use regex match):
While the answers that you can't parse HTML with regexes are correct, they don't apply here. The OP just wants to parse one HTML tag with regexes, and that is something that can be done with a regular expression.
The suggested regex is wrong, though:
<([a-z]+) *[^/]*?>
If you add something to the regex, by backtracking it can be forced to match silly things like <a >>, [^/] is too permissive. Also note that <space>*[^/]* is redundant, because the [^/]* can also match spaces.
My suggestion would be
<([a-z]+)[^>]*(?<!/)>
Where (?<! ... ) is (in Perl regexes) the negative look-behind. It reads "a <, then a word, then anything that's not a >, the last of which may not be a /, followed by >".
Note that this allows things like <a/ > (just like the original regex), so if you want something more restrictive, you need to build a regex to match attribute pairs separated by spaces.
As many people have already pointed out, HTML is not a regular language which can make it very difficult to parse. My solution to this is to turn it into a regular language using a tidy program and then to use an XML parser to consume the results. There are a lot of good options for this. My program is written using Java with the jtidy library to turn the HTML into XML and then Jaxen to xpath into the result.
Although it's not suitable and effective to use regular expressions for that purpose sometimes regular expressions provide quick solutions for simple match problems and in my view it's not that horrbile to use regular expressions for trivial works.
There is a definitive blog post about matching innermost HTML elements written by Steven Levithan.
Here's a PCRE regular expression for XML/XHTML, built from a simplified EBNF syntax definition:
/
(?(DEFINE)
(?<tag> (?&tagempty) | (?&tagopen) ((?&textnode) | (?&tag) | (?&comment))* (?&tagclose))
(?<tagunnested> (?&tagempty) | (?&tagopen) ((?&textnode) | (?&comment))* (?&tagclose))
(?<textnode> [^<>]+)
(?<comment> <!--([\s\S]*?)-->)
(?<tagopen> < (?&tagname) (?&attrlist)? (?&ws)* >)
(?<tagempty> < (?&tagname) (?&ws)* (?&attrlist)? (?&ws)* \/>)
(?<tagclose> <\/ (?&tagname) (?&ws)* >)
(?<attrlist> ((?&ws)+ (?&attr))+)
(?<attr> (?&attrunquoted) | (?&attrsinglequoted) | (?&attrdoublequoted) | (?&attrempty))
(?<attrempty> (?&attrname))
(?<attrunquoted> (?&attrname) (?&ws)* = (?&ws)* (?&attrunquotedvalue))
(?<attrsinglequoted> (?&attrname) (?&ws)* = (?&ws)* ' (?&attrsinglequotedvalue) ')
(?<attrdoublequoted> (?&attrname) (?&ws)* = (?&ws)* " (?&attrdoublequotedvalue) ")
(?<tagname> (?&alphabets) ((?&alphabets) | (?&digits))*)
(?<attrname>(?&alphabets)+((?&alphabets)|(?&digits)|[:-]) )
(?<attrunquotedvalue> [^\s"'=<>`]+)
(?<attrsinglequotedvalue> [^']+)
(?<attrdoublequotedvalue> [^"]+)
(?<alphabets> [a-zA-Z])
(?<digits> [0-9])
(?<ws> \s)
)
(?&tagopen)
/x
This illustrates how to build regular expressions for context-free grammars. You can match other parts of the definition by changing the match on the last line from (?&tagopen) to e.g. (?&tagunnested)
For XML/XHTML the consensus is no!
Credits to nikic for supplying the idea.
First, to answer the direct question: Your regex has a bug since it will exclude a tag with a slash anywhere, not just at the end. For example it would exclude this valid opening tag: <a href="foo/bar.html"> because it has a slash in an attribute value.
We can fix that, but more seriously, this regex will lead to false positives, because it will also match inside comments and cdata sections, where the same characters doesn't represent a valid tag. For example:
<!-- <foo> -->
or
<![CDATA[ <foo> ]]>
Especially html strings embedded in scripts is likely to trigger false positives, and so is the regular use of < and > as comparison operators in JavaScript. And of course sections of html which is commented-out with <!-- -->.
So to only match actual tags, you also need to be able to skip past comments and cdata sections. So you need the regex to also match comments and cdata, but only capture the opening tags. This is still possible using a rexep, but it becomes significantly more complex, for example:
(
<!-- .*? --> # comment
| <!\[CDATA\[ .*? \]\]> # CData section
| < \w+ ( "" [^""]* "" | ' [^']* ' | [^>/'""] )* /> # self-closing tag
| (?<tag> < \w+ ( "" [^""]* "" | ' [^']* ' | [^>/'""] )* > ) # opening tag - captured
| </ \w+ \s* > # end tag
)
And this just for XHTML conforming to the HTML compatibility guidelines. If you want to handle arbitrary XHTML you should also handle processing instructions and internal DTD's, since they can also embed false positives. If you also want to handle HTML there are additional complexities like the <script>-tag. And if you also want to handle invalid HTML it gets yet more complex.
Given the complexity, I would not recommend going down that road. Instead, look for an off-the-shelf (X)HTML parsing library which can solve your problem.
A parser typically uses regular expressions (or similar) under the hood to split the document into "tokens" (doctype, start tags, end tags, text content etc.). But someone else will have debugged and tested these regexes for you! Depending on the type of parser it may further build a tree structure of elements by matching start tags to end tags. This will almost certainly save you a lot of time.
The exact parser library to use depend on your language and platform and the task you are solving. If you need access to the actual tag-substrings (e.g. if you are writing a syntax highlighter for HTML) you need to use a SAX-style parser which exposes the syntax tokens directly.
If you are only performing the tag-matching in order to manually build a syntax tree of elements, then a DOM parser does this work for you. But a DOM parser does not expose the underlying tag syntax, so does not solve the exact problem you describe.
You should also consider if you need to to parse invalid HTML. This is a much more complex task, but on the wild web most of the HTML is actually invalid. Something like Pytons html5lib can parse invalid HTML.
RegEx match open tags except XHTML self-contained tags
All other tags (and content) are skipped.
This regex does that. If you need to match only specific Open tags, make a list
in an alternation (?:p|br|<whatever tags you want>) and replace the [\w:]+ construct
in the appropriate place below.
<(?:(?:(?:(script|style|object|embed|applet|noframes|noscript|noembed)(?:\s+(?>"[\S\s]*?"|'[\S\s]*?'|(?:(?!/>)[^>])?)+)?\s*>)[\S\s]*?</\1\s*(?=>)(*SKIP)(*FAIL))|(?:[\w:]+\b(?=((?:"[\S\s]*?"|'[\S\s]*?'|[^>]?)*)>)\2(?<!/))|(?:(?:/?[\w:]+\s*/?)|(?:[\w:]+\s+(?:"[\S\s]*?"|'[\S\s]*?'|[^>]?)+\s*/?)|\?[\S\s]*?\?|(?:!(?:(?:DOCTYPE[\S\s]*?)|(?:\[CDATA\[[\S\s]*?\]\])|(?:--[\S\s]*?--)|(?:ATTLIST[\S\s]*?)|(?:ENTITY[\S\s]*?)|(?:ELEMENT[\S\s]*?))))(*SKIP)(*FAIL))>
https://regex101.com/r/uMvJn0/1
# Mix html/xml
# https://regex101.com/r/uMvJn0/1
<
(?:
# Invisible content gets failed
(?:
(?:
# Invisible content; end tag req'd
( # (1 start)
script
| style
| object
| embed
| applet
| noframes
| noscript
| noembed
) # (1 end)
(?:
\s+
(?>
" [\S\s]*? "
| ' [\S\s]*? '
| (?:
(?! /> )
[^>]
)?
)+
)?
\s* >
)
[\S\s]*? </ \1 \s*
(?= > )
(*SKIP)(*FAIL)
)
|
# This is any open html tag we will match
(?:
[\w:]+ \b
(?=
( # (2 start)
(?:
" [\S\s]*? "
| ' [\S\s]*? '
| [^>]?
)*
) # (2 end)
>
)
\2
(?<! / )
)
|
# All other tags get failed
(?:
(?: /? [\w:]+ \s* /? )
| (?:
[\w:]+
\s+
(?:
" [\S\s]*? "
| ' [\S\s]*? '
| [^>]?
)+
\s* /?
)
| \? [\S\s]*? \?
| (?:
!
(?:
(?: DOCTYPE [\S\s]*? )
| (?: \[CDATA\[ [\S\s]*? \]\] )
| (?: -- [\S\s]*? -- )
| (?: ATTLIST [\S\s]*? )
| (?: ENTITY [\S\s]*? )
| (?: ELEMENT [\S\s]*? )
)
)
)
(*SKIP)(*FAIL)
)
>
