Save web scraped result to txt files by names

user_15669960 0
Viewed 46

I scraped a list of professor contact information from a school website, and now I want to save them individually by name, each name txt file contains their email, tel and office. ideal outcome

Currently my code is

from bs4 import BeautifulSoup as bs
import requests
url = 'https://www.cb.cityu.edu.hk/is/people/academic.html'
webpage = requests.get(url)
page = bs(webpage.content, 'html.parser')

#define list
name_list = []
phone_list = []
email_list = []
result = page.find_all('div', attrs = {'class': 'staff-details'})
for person in result:
    print(person.text)
beautifulsoup python
2 Answers

Open file with name you want using context manager in w+ mode here is a sample code for you

---> Inside Your For Loop
with open(file_name_come_here,"w+") as f :
      f.write(content_come_here_as_string)
user_110634720

You can use a loop to fetch the data and simultaneously save the data in a text file.

from bs4 import BeautifulSoup as bs
import requests
url = 'https://www.cb.cityu.edu.hk/is/people/academic.html'
webpage = requests.get(url)
page = bs(webpage.content, 'html.parser')
prof_list = page.select(".staff-details")


for i in prof_list:
    name = i.select_one('.name >a').text
    email = i.select_one('.list-info div.value:nth-child(2) > a').text
    phone = i.select_one('.list-info div.value:nth-child(4)').text
    office = i.select_one('.list-info div.value:nth-child(6)').text

    with open(name+'.txt', 'w+') as file:
        file.write("Email:\n")
        file.write(email)
        file.write('\nPhone:\n')
        file.write(phone)
        file.write("\nOffice\n")
        file.write(office)
user_141197820
Related