Scraping <div> inside a <div>

user_15421828 0
Viewed 101

I'm having some trouble scraping the names of a <div> that are already in a <div> (It works with complete other part even though I tried to search for a specific card-body)

https://namemc.com/minecraft-names?sort=asc&length_op=&length=3&lang=&searches=500

I need this part:

<div class="card-body p-0">
   <div class="row no-gutters py-1 px-3">
      <div class="col col-lg order-lg-1 text-nowrap text-ellipsis">
         <a href="/name/example" translate="no">example</a>

Even though I find names, they are not from the list I want. Does anybody know how to locate them? Im using beautifulsoup and lxml. Part of my code:

from bs4 import BeautifulSoup
import requests
html_text = requests.get('https://namemc.com/minecraft-names?sort=asc&length_op=&length=3&lang=&searches=500').text
soup = BeautifulSoup(html_text, 'lxml')
itemlocator = soup.find('div', class_='card-body p-0')
for items in itemlocator:
print(items)
web-scraping beautifulsoup python-3.x html python
1 Answers

The following script should produce the available names that you see in that page. However, it seems you are only after the container in which Commander is available. In that case, you can try like below to get the desired portion which is concise and efficient compare to your current attempt.

import requests
from bs4 import BeautifulSoup

link = 'https://namemc.com/minecraft-names?sort=asc&length_op=&length=3&lang=&searches=500'

headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36'
}

html_text = requests.get(link,headers=headers)
soup = BeautifulSoup(html_text.text,'lxml')
item = soup.select_one(".card-body > .no-gutters a[href^='/name/Commander']")
item_text = item.get_text(strip=True)
datetime = item.find_parent().find_parent().select_one("time").get("datetime")
print(item_text,datetime)

Output:

Commander 2021-03-19T13:10:40.000Z
user_133540030
Related