Trying to get all links from string which ends with ','

user_8113828 0
Viewed 80

I need to get all links with albums of artist from Genius page. All links can be found in fifth tag preload-content value of attribute data-preload_data. This value store in var string as str.

I tried to extract all links start with 'https://genius.com/albums/' and end with ',' but it didn't work. When i didn't use $ sign for ending i got right amount of links but without necessary ending part

import urllib.request, urllib.parse, urllib.error
from bs4 import BeautifulSoup
import ssl
from urllib.request import Request, urlopen
import re

name = input('Rapper - ')
url = 'https://genius.com/artists/'+name+''
hdr = {'User-Agent': 'Mozilla/5.0'}
req = Request(url,headers=hdr)
html = urlopen(req)
soup = BeautifulSoup(html, 'html.parser')

hrefs = soup.find_all("preload-content")
string = hrefs[5]['data-preload_data']

result = re.findall('(https://genius.com/albums/'+name+'.,$)', string)
print(result)
beautifulsoup python regex
1 Answers

You can use

re.findall(r'(https://genius\.com/albums/' + re.escape(name) + '/[^"\'\s<>]*?)&quot;,', string)

See the regex demo.

Details

  • (https://genius\.com/albums/' + re.escape(name) + '/[^"\'\s<>]*?) - Group 1:
    • https://genius\.com/albums/' + re.escape(name) + '/ - a literal substring
    • [^"\'\s<>]*? - any zero or more chars other than ", ', whitespace, < and >, as few as possible (due to *? lazy quantifier)
  • &quot;, - a literal string

Note the name when used in a regex must have all special chars escaped for the regex to be syntactically correct, hence the use of re.escape(name).

user_38329700
Related