How can I clean html code to return only the number values?

Question

,
,
,
,
,

I currently have the above html code that is in a list. I wish to use python so that it may output the following and then append to a list:

user_13825434 · Answer

I would recommend using Beautiful Soup which is a very popular html parsing module that is uniquely suited for this kind of thing. If each element has the attribute of title then you could do something like this:

from bs4 import BeautifulSoup
import requests
def randomFacts(url):
    r = requests.get(url)
    bs = BeautifulSoup(r.content, 'html.parser')
    title = bs.find_all('div')
    for each in title:
        print(each['title'])

Beautiful Soup is my normal go to for html parsing, hope this helps.

user_10292330 · Answer

Here are 3 possibilities. In the first 2 versions we make sure the class checks out before appending it to the list - just in case there are other divs that you don't want to include. In the third method there isn't really a good way to do that. Unlike adrianp's method of splitting, mine doesn't care where the title is.

The third method may be a bit confusing so, allow me to explain it. First we split everywhere that title=" appears. We dump the first index of that list because it is everything before the first title. We then loop over the remainder and split on the first quote. Now the number you want is in the first index of that split. We do an inline pop to get that value so we can keep everything in a list comprehension, instead of expanding the entire loop and wrestling the values out with specific indexes.

To load the html remotely, uncomment the commented html var and replace "yourURL" with the proper one for you.

I think I have given you every possible way of doing this - certainly the most obvious ones.

from bs4 import BeautifulSoup
import re, requests

html = ' \
 \
 \
 \
'

#html = requests.get(yourURL).content

# possibility 1: BeautifulSoup

soup = BeautifulSoup(html, 'html.parser')

# assumes that all bb-fl classed divs have a title and all divs have a class
# you may need to disassemble this generator and add some extra checks
bs_titleval = [div['title'] for div in soup.find_all('div') if 'bb-fl' in div['class']]
  
print(bs_titleval)


# possibility 2: Regular Expressions ~ not the best way to go
# this isn't going to work if the tag attribute signature changes

title_re = re.compile('', re.I)

re_titleval = [m.group(1) for m in title_re.finditer(html)]
    
print(re_titleval)


# possibility 3: String Splitting ~ 
# probably the best method if there is nothing extra to weed out

title_sp = html.split('title="')
title_sp.pop(0) # get rid of first index

# title_sp is now ['10">...', '3">...', '18">...', etc...]
sp_titleval = [s.split('"').pop(0) for s in title_sp]

print(sp_titleval)

user_3480910 · Answer

Assuming that each div is saved as a string in the variable div, you can do the following:

number = div.split()[3].split('=')[1]

Each div should be in the same format for this to work.

How can I clean html code to return only the number values?

3 Answers