How to position a whole text of such a whole div with Xpath?

user_18107761 0
Viewed 38

My test website is https://www.nature.com/articles/ncomms12114.

I want to get the text from the "Additional Information" part.

My Xpath code is

from lxml import etree
page_text = requests.get(url="https://www.nature.com/articles/ncomms12114",headers = headers).text

tree = etree.HTML(page_text)
addInformatin_raw= tree.xpath("//h2[text()='Additional information']/..")[0]
addInformatin = addInformatin_raw.xpath('./div//text()')[0]

But the response of my code is just a string called-"How to cite this article:" which is part of the text.

My ideal response is"How to cite this article: Wang, S.Y. et al. Hypoxia causes transgenerational impairments in reproduction of fish. Nat. Commun. 7:12114 doi: 10.1038/ncomms12114 (2016)."

So I want to know how to revise my code.Thank you foe your answer.

xpath python
1 Answers

I think you should remove the [0] here.
//h2[text()='Additional information']/..//div XPath gives the element containing exactly what you are looking for.
So, I'm not familiar with etree but it seems like your code could be something like this:

from lxml import etree
page_text = requests.get(url="https://www.nature.com/articles/ncomms12114",headers = headers).text

tree = etree.HTML(page_text)
addInformatin_texts= tree.xpath("//h2[text()='Additional information']/..//div//text()")

this will give you 6 texts.

user_34854340
Related