How to use XMLStarlet / Xpath to select text inside <div> but exclude some inner <span>

user_104897 0
Viewed 288

I have such html files. Basically it has div containing texts with one inner span and the rest text part has pretty arbitrary format.

<html>
<div>
<span class="c1">Text1</span><br/>
Text4<br/>
Text5
</div>
<div>
<span class="c1">TextA</span><a href="...">TextD</a>
</div>
</html>

it is trivial to select/print only specific text inside span with xml sel -t -m "/html/div" -v "span[@class='c1']" -n

However I don't know how to select/print the rest text within but outside the span regardless of any other tags like <br/>. The function text() does not work as I expect.

xml sel -t -m "/html/div" -v "concat(span[@class='c1'],'|',text(),'$')" -n will cut text behind <br/> tags.

how can I get something like

Text1|
Text4
Text5$
TextA|TextD$
xmlstarlet xpath xml text
4 Answers

It's been a while but you said,

I would like to know if there is a better way than substr method

Here's a version which produces output in document order by using a template matching the union of desired elements inside an /html/div. Other elements than br are output with their text value followed by a vertical bar. Each br element is output with a newline and the normalized text following br. Finally, after 2 -bs breaking the -m nesting, each div is ended by a dollar sign followed by a newline.

xmlstarlet sel -T -t \
-m "/html/div" \
  -m 'a | br | span[@class="c1"]' \
  --if 'local-name() = "br"' \
       -n -v 'normalize-space(following-sibling::text())' \
  --else -v '.' -o '|' \
  -b -b -o '$' -n  \
source

Output:

Text1|
Text4
Text5$
TextA|TextD|$

To strip the last | before a $ replace -o '|' with -v 'substring("|",1,count(following-sibling::*))' which emits | only if there are more sibling elements.

To add support for p elements, for example, append | p to the 2nd -m expression and add <p>Hello from P</p> to an /html/div in the source file.

(Documentation on --if … --elif … --else … is, er, less than copious but as far as I can make out, from XSLT code output by -C option, the --if clause is terminated by -b.)

I used xmlstarlet 1.6.1.

user_144033690

I've tested some xpath's so the best one which I found is

//div/descendant-or-self::*/text()[normalize-space()]

xpath result

It indicates the context node and all of its descendants, get text values that aren't empty.

about XPath axis

user_74948900

I think there is a better way to do the job, but I have this one. I have tested it in scrapy xpath.I think its help you.

print(data.xpath("concat(string(//div[1]/span[contains(@class, 'c1')]),'|', '\n',substring-before(substring-after(//div[1], 'Text1'), 'Text5'),'\n',substring-after(substring-after(//div[1], 'Text1'),'Text4'),'$', '\n',//div[2]/span[contains(@class, 'c1')]/text(),'|',//div[2]/a/text(),'$')").get())

output:

Text1|
Text4
Text5$
TextA|TextD$
user_48351220

Somehow I found a solution with substring-after

xml sel -t  -m "/html/div" -v "span[@class='c1']" -o '|' -v "substring-after(.,span[@class='c1'])" -n

but I would like to know if there is a better way than substr method.

user_1048970
Related