beautifulsoup - filter text of anchor tag is not working when there is img tag in front of text

user_4187091 0
Viewed 74

I have the following html content :

from bs4 import BeautifulSoup
import re

html = """<a href="http://app_url1" >install app xyz</a>
<a href="http://app_url2" ><img src="/path.jpg">install app xyz</a>
<a href="http://app_url3" >install app aaa</a>
<a href="http://app_url4">install app aaa</a>"""
soup = BeautifulSoup(html, "html.parser")
print(soup.findAll("a", text=re.compile("xyz$")))

I want to filter the anchor tag texts that end with a given regex pattern (like xyz here)? I am looking to pass a regex pattern to findAll instead of extra iteration of all anchor tags. But I am getting output only one anchor tag as

<a href="http://app_url1">install app xyz</a>

The other anchor tag which has img in front of text is getting ignored

expected output:

<a href="http://app_url1" >install app xyz</a>
<a href="http://app_url2" ><img src="/path.jpg">install app xyz</a>
beautifulsoup python-3.x html regex
2 Answers

Filter by only text=re.compile("xyz$") then use .parent

Ex:

from bs4 import BeautifulSoup
import re

html = """<a href="http://app_url1" >install app xyz</a>
<a href="http://app_url2" ><img src="/path.jpg">install app xyz</a>
<a href="http://app_url3" >install app aaa</a>
<a href="http://app_url4">install app aaa</a>"""
soup = BeautifulSoup(html, "html.parser")
result = [el.parent for el in soup.findAll(text=re.compile("xyz$"))]
print(result)

Output:

[<a href="http://app_url1">install app xyz</a>, <a href="http://app_url2"><img src="/path.jpg"/>install app xyz</a>]
user_5323120

You can use CSS selector select instead of extra iteration of all anchor tags.
Example:

from bs4 import BeautifulSoup
import re

html = """<a href="http://app_url1" >install app xyz</a>
<a href="http://app_url2" ><img src="/path.jpg">install app xyz</a>
<a href="http://app_url3" >install app aaa</a>
<a href="http://app_url4">install app aaa</a>"""
soup = BeautifulSoup(html, "html.parser")
print(soup.select('a:contains("xyz")'))

Output will be:

[<a href="http://app_url1">install app xyz</a>, <a href="http://app_url2"><img src="/path.jpg"/>install app xyz</a>]

For getting href content from the list of the above output:

anchors = soup.select('a:contains("xyz")')
href = [i['href'] for i in anchors]
print(href)

Output will be:

['http://app_url1', 'http://app_url2']
user_95015080
Related