How can I iterate over multiple tags in soup.findAll('tag1', 'tag2', 'tag3')?

user_12955644 0
Viewed 133

I'm trying to write a python script where modifying certain tags in multiple html files will be automated; running single command from the terminal.

I constructed the code base.

In my code base something I've done like below. Is there even more convenient way to do so with less code?

#modifying the 'src' of <img> tag in the soup obj
for img in soup.findAll('img'):
    img['src'] = '{% static ' + "'" + img['src'] + "'" + ' %}'

#modifying the 'href' of <link> tag in the soup obj
for link in soup.findAll('link'):
    link['href'] = '{% static ' + "'" + link['href'] + "'" + ' %}'

#modifying the 'src' of <script> tag in the soup obj
for script in soup.findAll('script'):
    script['src'] = '{% static ' + "'" + script['src'] + "'" + ' %}'

For instance, can I do it in single for loop instead of 3? Not saying it has to be like the way I wrote below, any good practice suggestion is what I'm looking for.

for img, link, script in soup.findAll('img', 'link', 'script'):
    rest of the code goes here....
beautifulsoup automation html python performance
2 Answers

Perhaps use a dictionary to retrieve appropriate attribute? Also, use faster css selectors.

import requests
from bs4 import BeautifulSoup as bs

r = requests.get('https://stackoverflow.com/questions/66541098/how-can-i-iterate-over-multiple-tags-in-soup-findalltag1-tag2-tag3')
soup = bs(r.content, 'lxml')

lookup = {
    'img':'src',
    'link': 'href',
    'script':'src'
}

for i in soup.select('img, link, script'):
    var = lookup[i.name]
    if i.has_attr(var):
        i[var] = '{% static ' + "'" + i[var] + "'" + ' %}'
        print(i[var])
user_62412350

Yes you can. You can pass a list of elements to findAll method

for element in soup.findAll(['img', 'link', 'script']): # use find_all for bs4
    
    if element.name == 'img':
        value = element['src']
    elif element.name == 'href':
        value = element['href']
    elif element.name == 'script':
        value = element['src']
    else:
        continue
        
    print(val)
user_65033290
Related