Filter strings scraped from input form in Python

Viewed 26

How do I filter out certain skills like 'django' and 'Django' from a collection of skills provided by users through an input form using a Python function?

I've requests and bs4 to get the raw data, but I need to filter through the results. Here's my code so far:

from bs4 import BeautifulSoup
import requests
import time

unfamiliar_skills = list(map(str,input('>')))
def find_jobs():
    html_text = requests.get('https://www.timesjobs.com/candidate/job-search.html?searchType=personalizedSearch&from=submit&txtKeywords=python&txtLocation=').text
    soup = BeautifulSoup(html_text, 'lxml')
    jobs = soup.find_all('li', class_ = 'clearfix job-bx wht-shd-bx')
    jobs = soup.find_all('li', class_ = 'clearfix job-bx wht-shd-bx')
    # we first created the parsing for one output, then used for loop to parse multiple instances of that.
    for index, job in enumerate(jobs):
        published_date = job.find('span', class_ = 'sim-posted').span.text # must be b 1st to prevent scraping if the pub date is not == few days ago
        if 'few' in published_date: 
            company_name = job.find('h3', class_ = 'joblist-comp-name').text.replace(' ','')
            skills = job.find('span', class_ = 'srp-skills').text.replace(' ','')
            more_info = job.header.h2.a['href'] # like in a dictionary
            
            if filter(unfamiliar_skills, skills):
                with open(f'C:/Users/USER/{index}.txt', 'w') as f:
                    f.write(f'Company Name: {company_name.strip()} \n')
                    f.write(f'Required Skills: {skills.strip()} \n')
                    f.write(f'more_info: {more_info} \n')    
                print(f'File saved: {index}')
if __name__ == '__main__':
    while True:
        find_jobs()
        time_wait = 10
        print(f'Waiting {time_wait} minutes...')
        time.sleep(time_wait*60)

Here is the printed output of skills variable:

rest,python,database,django,debugging,mongodb

python,webtechnologies,linux,mobile,mysql,angularjs,javascript

rest,python,security,debugging

python,docker,messaging,pythonscripting

python,git,django

python,database,django,mysql,api

python,hadoop,machinelearning

rest,python,django,git

python,django,,framework

python,java,scala

python,linux,windows,sql

python,webdeveloper,webservices

rest,python,database,django,api

Python,Django,Flask

python,django,javascript,webprogramming

python,Django,ObjectRelationalMapper

python,webtechnologies,webtechnologies

python,django,html5,javascript

python,django,html5,javascript

None
0 Answers
Related