Webscraping from word hippo

user_5176763 0
Viewed 286

I have a query regarding word scraping from word hippo. I am new to Beautiful Soup and don't how to get this list of words (described below)

I am trying to get all the synonyms under the following sections of the page (Even those synonyms below the 'more' button.

Section 1

enter image description here

Section 2

enter image description here

#My code.
import requests
from bs4 import BeautifulSoup

response = requests.get("https://www.wordhippo.com/what-is/another-word-for/guard.html")
soup = BeautifulSoup(response.content, 'html.parser')

# select only first '<section class="synonyms-container....'
synonyms = soup.select('.MainContentContainer > section > .synonyms-container a')
print ('synonyms for: Guard')
for synonym in synonyms:
            print (synonym.text)

Please help extract a list of these words.

Thanks in advance.

web-scraping beautifulsoup python-3.x
2 Answers

The words is in .relatedwords class container, to get the first and second section loop it twice.

synonyms = soup.select('.relatedwords')
for i in range(0, 2):
    print ('synonyms section ' + str(i + 1))
    print (synonyms[i].text)

if you want to store each word as list use synonyms[i].split("\n")

user_40823440

Try this:

import requests
from bs4 import BeautifulSoup

page = requests.get("https://www.wordhippo.com/what-is/another-word-for/guard.html")
soup = BeautifulSoup(page.content, 'html.parser')

synonyms = [
    a.getText() for a in soup.select_one('div.relatedwords').find_all("a")
]

for synonym in synonyms:
    print(synonym)

Output:

guardian
custodian
warden
keeper
sentry
escort
lookout
watch
minder
protector
... and more
user_61067910
Related