Part of the html that I am scraping looks like this:
<h2> <span class="headline" id="Profile">Profile</span></h2>
<ul><li> <b>Name</b> Albert Einstein
</li><li> <b>Birth Name:</b> Alberto Ein
</li><li> <b>Birthdate:</b> December 24, 1986
</li><li> <b>Birthplace:</b> <a href="/Ulm" title="Dest">Ulm</a>, Germany
</li><li> <b>Height:</b> 178cm
</li><li> <b>Blood Type:</b> A
</li></ul>
I want to extract each component - so name, birth name, birthday, etc.
To extract the name I do:
a_name = response.xpath('//ul/li/b[contains(text(),"Name")]/../descendant::text()').extract()
then I check that a_name is not an empty list and I call:
"".join(a_name[2:]).strip()
I do this for consistency since in Birthplace, I just want to extract the text, excluding all the html attributes. So I would get Ulm, Germany.
The problem is that when I use contains(text(), "Name"), the entry for Birth Name also matches. How can I avoid this when building my selector?
With a regular expression I could specify something like text() matches ^Name.* since the text Name may or may not be followed by a colon and or space.
Is there a way to use regular expressions to solve this problem?