Here's the code I'm using:
def get_rank(keyword, country="in", language="en"):
urls=[]
r=requests.get(f"https://www.google.com/search?q={keyword}&gl={country}&hl={language}")
soup=BeautifulSoup(r.text, 'lxml')
urls=[re.search("(http[s]?://[a-z0-9-.]*)",ele.a['href']).group(1) for ele in soup.find_all('div', class_="Gx5Zad fP1Qef xpd EtOod pkphOe")]
return urls
Output:
https://apple.com
Required Output:
https://apple.com/something/