Python + BeautifulSoup: Finding a HTML tag where an attribute contains a matched pattern of text?

user_4744661 0
Viewed 601

I'm new to both Python and BeautifulSoup. I'm trying to figure out how to match only the tags that are <div> elements that contain a certain matched pattern of text belonging to an attribute. For example, all cases where the 'id' : 'testid', or everywhere the 'class' : 'title'.

This is what I have so far:

def cleanup(filename):
    fh = open(filename, "r")

    soup = BeautifulSoup(fh, 'html.parser')

    for div_tag in soup.find('div', {'class':'title'}):
        h2_tag = soup.h2_tag("h2")
        div_tag.div.replace_with(h2_tag)
        del div_tag['class']

    f = open("/tmp/filename.modified", "w")
    f.write(soup.prettify(formatter="html5"))
    f.close()

Once I can match all those particular elements, at that point I can figure out how to manipulate the attributes (delete the class, rename the tag itself from <div> to <h1>, etc). So I'm aware that actual part of the cleanup probably doesn't work with how it is currently.

beautifulsoup html python
2 Answers

It seems this works sufficiently, but let me know if there's a "better" or "more standard" way to do it.

for tag in soup.findAll(attrs={'class':'title'}):
    del tag['class']
user_47446610

.find(tagName, attributes) return single element

.find_all(tagName, attributes) return multiple element (list)

more you can find it in the doc

to replace you need to create element .new_tag(tagName) and to delete attribute del element.attrs[attributeName] see below for example

from bs4 import BeautifulSoup
import requests

html = '''
<div id="title" class="testTitle">
  heading h1
</div>
'''
soup = BeautifulSoup(html)

print 'html before'
print soup

div = soup.find('div', id="title")

#delete class attribute
del div.attrs['class']

print 'html after remove attibute'
print soup

# to replace, create h1 element
h1 = soup.new_tag("h1")
# set text from previous element
h1.string = div.text
# uncomment to set ID
# h1['id'] = div['id']
div.replace_with(h1)

print 'html after replace'
print soup
user_4582140
Related