Is there a way to find all indexes of a certain tag in a string relatively to the same string with removed tags?

user_8661764 0
Viewed 108

I have an HTML string, with several <em>...</em> tags in it. I need to find all the indexes of these tags relatively to the string, where all the tags are removed.

For example:

from bs4 import BeautifulSoup


string = "<em>This</em> is <em>a sample</em> string"
string_without_tags = BeautifulSoup(string, "lxml").text

# [(0, 4), (8, 16)] <=> "This" and "a sample"
print(string_without_tags[:4], ", ", string_without_tags[8:16], sep="")

I think I could just use a loop, but maybe there is more efficient way to do what I need?

web-scraping beautifulsoup python
2 Answers

You can use .find_all(text=True).

For example:

from bs4 import BeautifulSoup

string = "<em>This</em> is <em>a sample</em> string"
soup = BeautifulSoup(string, 'html.parser')

cur, idxs = 0, []
for t in soup.find_all(text=True):
    if t.parent.name == 'em':
        idxs.append((cur, cur+len(t)))
    cur += len(t)
print(idxs)

Prints:

[(0, 4), (8, 16)]
user_100359850

If, as your question hints at, you need the indexes only as a step towards extracting the the text from the 'em' tags, you can simply skip that step and extract the text direct.

from bs4 import BeautifulSoup

string = "<em>This</em> is <em>a sample</em> string"
soup = BeautifulSoup(string, 'html.parser')
print(", ".join([x.get_text() for x in soup.find_all("em")]))

Prints:

This, a sample
user_8395400
Related