Find CSS selector for a element using beautifulsoup python

Viewed 20

I need to find automatically 'next button' using python. I thought of two solutions for this..( i will input first chp link, second page link) .I need to do it via packages other than selenium since i can't add selenium in my requirement..

  1. I found all the css selectors in the first page and compare the href to second link so I can find the next page css selector. I couldn't find any way to get the css selectors for a element in python without using selenium
  2. I can find the element in python now. but couldn't find the css selector for it. Is there any way to get css selector from element..

PS: this code has to work in many sites as much as possible . Its only job is to find the selector for the button or element which is redirecting to second page. So I can continue do so and move to third and 4th and so on with the css selector i got from first page.(i got the xpath though but xpath sometimes changes in site)

I found the element using this code.

try:
            response = requests.get(firstchplink, headers=headers, timeout=10)
        except:
            pass
        response.encoding = response.apparent_encoding
        soup = BeautifulSoup(response.content, 'html5lib')
        htm = response.text
        sel = parsel.Selector(htm)
       
        urls = sel.css('a ::attr(href)').extract()

        psrt = ''
        for url in urls:
                full_url = urljoin(firstchplink, url)
                if full_url == secondchplink:
                    psrt = url

here i found the element using soup.

href = [i for i in soup.find_all("a") if i.get("href") == psrt]

please help me in finding the css selector from the page element found by soup

0 Answers
Related