Select all tag (and extract text) between two other tags with BeatifulSoup

user_9882202 0
Viewed 375

I would like to extract all instances of a given tag that are included between two tags. Currently I am working with BeautifulSoup. Below you can find an example:

<p class='x' id = '1'> some content 1 <p>
<p class='y' id = 'a'> some content a <p>
<p class='y' id = 'b'> some content b <p>
<p class='y' id = 'c'> some content c <p>
<p class='potentially some other class'> <p>
<p class='x' id = '2'> some content 2 <p>
<p class='y' id = 'd'> some content d <p>
<p class='y' id = 'e'> some content e <p>
<p class='y' id = 'f'> some content f <p>

I am interested in selecting all the instances of class 'y' between the two tags 'x', which also have different id's. With regards to the specific example, I would like to select all p's with class = 'y' to then retrieve the text. My final desired output would be: 'some content a', 'some content b', and 'some content c'.

I tried using the findAllNext method, but that gives me 'some content a', 'some content b', 'some content c' AND 'some content d', 'some content e', 'some content f'.

Below is my code

par = BeautifulSoup(HTML_CODE).content, 'lxml') 
loc = par.find('p', class_ = 'x', id ='1')
desired = loc.findAllNext('p', class_ = 'y')

Is there any way to avoid selecting also the instances of class = 'y' that appear after the tag with class='x' with id = '2' ?

Thank you.

beautifulsoup tags html python
1 Answers

You can start iterating from where you want and end it until found something mark finished.

from bs4 import BeautifulSoup

html = """

<p class='x' id = '1'> some content 1 </p>
<p class='y' id = 'a'> some content a </p>
<p class='y' id = 'b'> some content b </p>
<p class='y' id = 'c'> some content c </p>
<p class='potentially some other class1'> potentially some other class 1 </p>
<p class='potentially some other class2'> potentially some other class 2</p>
<p class='potentially some other class3'> potentially some other class 3 </p>
<p class='x' id = '2'> some content 2 </p>
<p class='y' id = 'd'> some content d </p>
<p class='y' id = 'e'> some content e </p>
<p class='y' id = 'f'> some content f </p>
"""

soup = BeautifulSoup(html,"lxml")
start = soup.find("p",class_="y",id="c")
end = soup.find("p",class_="x",id="2")
def next_ele(ele,result=[]):
    row = ele.find_next("p")
    if not row or row == end:
        return result
    result.append(row)
    return next_ele(row,result)

print(next_ele(start))
user_93806940
Related