Python parsing web pages problem not completing the enitre loop

Viewed 12

Basically, I am trying to open a website and find the links inside it. Then, I want
to open the link that meets my criteria( order of the link given by the user) and to repeat the process until the number of attempts( an input by the user) is finished
this is the Code I am using:

import urllib.request, urllib.parse, urllib.error
from bs4 import BeautifulSoup
import ssl

# Ignore SSL certificate errors
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
su3=0
su=0
url = input('Enter Url ')
url2=url
position=input("Enter p")
attempts=input("Enter attempts:")
while su3<int(attempts):
 su3=su3+1
 html = urllib.request.urlopen(url2, context=ctx).read()
 soup = BeautifulSoup(html, 'html.parser')
 tags = soup('a')
 for tag in tags:
     su+=1
     if su==int(position):
      link=tag.get('href', None)
      print(link)
      url2=link
      print(url2)

The problem is that the loop is not repeating and it is just giving me the first link and does not go on with the loop. How can I fix it so the loops completes the task.
Thank you for your time. I apologize if something is unclear I am still new to this coding industry.

0 Answers
Related