Getting lyrics of song from genius lyrics with beautifulsoup │python 3.8

user_12105861 0
Viewed 656

I am trying to get the lyrics for a song from genius lyrics using beautifulsoup but when trying to print out the lyrics, I get no output. Here is my code:

import requests 
from bs4 import BeautifulSoup
songURL = requests.get("https://genius.com/Marshmello-and-bastille-happier-lyrics")
song = songURL.content
soup = BeautifulSoup(song, 'lxml')
lyrics = soup.find_all("section")
for lyr in lyrics:
    for lyr1 in lyrics.select("p"):
        print(lyr1.text)

Why is this not working, can somebody please look into this as I have been trying to do this for a while now.

python-requests beautifulsoup python-3.x html python
4 Answers

It seems that the server returns two version of the page: in one version there are tags with class="song_body-lyrics", in other version with class="Lyrics__Container...".

This script tries to handle both cases:

import requests 
from bs4 import BeautifulSoup

url = 'https://genius.com/Marshmello-and-bastille-happier-lyrics'
soup = BeautifulSoup(requests.get(url).content, 'lxml')

for tag in soup.select('div[class^="Lyrics__Container"], .song_body-lyrics p'):
    t = tag.get_text(strip=True, separator='\n')
    if t:
        print(t)

Prints:

[Intro]
Lately, I've been, I've been thinking
I want you to be happier, I want you to be happier
[Verse 1]

...and so on.
user_100359850 
import requests 
from bs4 import BeautifulSoup
songURL = requests.get("https://genius.com/Marshmello-and-bastille-happier-lyrics")
song = songURL.content
soup = BeautifulSoup(song, 'lxml')
final_lyrics = []
lyrics = soup.find('div', {'class': "lyrics"})
lyrics = lyrics.find_all('p')
for i in lyrics:
    final_lyrics.append(i.text)
    print(i)
user_98978440

you should get all texts those are in a specific div. you can find that specific div with devtools or viewsource in your browser. here that specific div is <div class='lyrics'> the unique feature of this div is its class, that is class 'lyrics' so we should find this specific div in HTML and then print all texts in that div.

import bs4 as bs
import urllib.request

source = urllib.request.urlopen('https://alirezaarabi.com/view-source_https___genius.com_Alessia-cara-ready-lyrics.html').read()

soup = bs.BeautifulSoup(source,'lxml')
print(soup.title.string)

for div in soup.find_all('div', class_='lyrics'):
    print(div.text)
user_108591140

If you take a look at the actual HTML source code, there are no section tags. Here's what the structure actually looks like:

<div class="song_body column_layout" initial-content-for="song_body">
  <div class="column_layout-column_span column_layout-column_span--primary">
    <div class="song_body-lyrics">
      
        <h2 class="text_label text_label--gray text_label--x_small_text_size u-top_margin">Happier Lyrics</h2>
      
      <div initial-content-for="lyrics">
        <div class="lyrics">
          
            <!--sse-->
            <p>[Intro]<br>
Lately, I've been, I've been thinking<br>
I want you to be happier, I want you to be happier<br>
<br>
...
user_109874320
Related