Extract all text from html after a specific tag?

user_12062508 0
Viewed 223

I want to extract the text of an HTML file after the second occurrence of a specific tag.

I already tried regex and bs4, but I can't figure out what is going wrong. Regex always gives me only the hit itself without the rest of the html file and bs4 just does not work, because I don't know how to specify the end of the file for.

Simplified:

<html>
    <veryspecific tag>
       abc
    </veryspecific tag>

    <stuff that comes before>
    </stuff that comes before>
    <...

       <veryspecific tag>
       abc
       </veryspecific tag>

       <other tags that come after>
       something
       </other tags that come after>
    </...>

    <other tags that come after2>
    something
    </other tags that come after2>
</html>

#I tried splitting it, so I can take the last part which should contain the end of the file, starting from the latest occurrence, but it did not work:

htmltxt.split(r'abc.*$')


# I also tried to get the last tag and try to "while" over the 2 to get the text:

last_tag = html_parsed.findall('a')[-1]

while specific_tag != last_tag:
   text = ...
   specific_tag = specific_tag.next

I find the desired tag and can extract it, but I also need the rest of the file. Is there an easy and pythonic way to do it?

beautifulsoup html python
1 Answers

Here's a suggestion using BeautifulSoup:

mark = soup.find('veryspecific').find_next('veryspecific')
all_other_tags = mark.find_all_next(name=True)

print(''.join(i.text for i in all_other_tags))

It gives me this output:

       something

    something
user_117748080
Related