Web scraping company description from StackOverflow companies

user_14577337 0
Viewed 172

I am trying to take the description of all the companies (all the pages) from https://stackoverflow.com/jobs/companies using beautifulsoup and having trouble figuring out if and how to use the "mt8 mb0 fs-body1 fc-black-700" class

from bs4 import BeautifulSoup
import requests

page = requests.get('https://stackoverflow.com/jobs/companies')

soup = BeautifulSoup(page.text, "html.parser")


descriptions = soup.find_all('p', class_ = 'mt8 mb0 fs-body1 fc-black-700')

for description in descriptions:
    print(description)
    print()
python-requests web-scraping beautifulsoup python
1 Answers

Yes. You can use mt8 mb0 fs-body1 fc-black-700 as the class name. This is how your final code should look like:

from bs4 import BeautifulSoup
import requests

r = requests.get('https://stackoverflow.com/jobs/companies').text

soup = BeautifulSoup(r,'html5lib')

descriptions = soup.find_all('p', class_ = 'mt8 mb0 fs-body1 fc-black-700')

for desc in descriptions:
    print(desc.text)

Output:

Verys was founded in 2012 by Mike Zerkel and Chris Antonius to help bring cutting edge technology innovation back to the United States – p...
We are a global team with plenty ofenterpriseand startup experience. Mix of creative, strategists, technologists, and engineer...
You’ll work with a talented and highly-motivated team that is friendly, fast-moving, self-managing, and highly capable with a sense of hum...
We take the long view of software development and maintenance by
About OutMatch
The OutMatch Talent Intelligence Platform brings clarity to talent decisions by gat...
We do way more than the usual software full stack development. We are also mechanical, electronics hardware, and firmware engineers: Angular.js...
At Citizens, we’re more than a network of brick and mortar branches. We’re a digital company that happens to be a bank.
We h...
Kaliber Labs is a full stack artificial intelligence company focused on bringing objectivity and consistency to the surgical setting with...
Our technologists work on a diverse range of solutions that include strategic technology initiatives, big data, mobile, electronic payments, mac...
We believe in cross-functional, autonomous teams owning strategic parts of the product, end-to-end.
A career at Paige is deeply mission-...
user_123098080
Related