How to scrape links inside td tags in python

user_19005568 0
Viewed 82

Here is the html code that I found from a website. I want the vk link which is found inside the td tag.

I tried so many ways in python to scrape that link but it always shows some type of error and sometimes it shows different links.

<thead>
<tr class="footable-header">
<th scope="col" 
class="ninja_column_0 
ninja_clmn_nm_date ">Date</th><th scope="col"class="ninja_column_1ninja_clmn_nm_download">download</th></tr></thead><tbody><tr data-row_id="0" 
class="ninja_table_row_0 nt_row_id_0"><td>01-05-2022</td><td>https://vk.com/doc722551386_632783806? hash=gjIfCA0ILqZ1LQlzftCyxZ4zOATANYnUqZXiZ1vsAJH&dl=5wFKrFiIzvVfYJ6M4m1z9ALqKzGdXJdsGAXv1NaBtSg</td> </tr>

Here is the python code that I tried:

import requests
from bs4 import BeautifulSoup

url="https://www.careerswave.in/dainik-jagran-newspaper-download/"
reqs = requests.get(url)
soup = BeautifulSoup(reqs.text,'html.parser')
f = open("vkdain.txt", "w")
for link in soup.find_all("a"):
data = link.get('href')
print(data)
web-scraping beautifulsoup python
1 Answers

If you just want to get the links in td this worked for me:

import requests
from bs4 import BeautifulSoup

url = "https://www.careerswave.in/dainik-jagran-newspaper-download/"
reqs = requests.get(url)
soup = BeautifulSoup(reqs.text, 'html.parser')
f = open("vkdain.txt", "w")
for link in soup.find_all("td"): # find all the td's
    if link.text.startswith('https://vk'): # check if the pattern is the one you want
        print(link.text)

This gets you the following result:

https://vk.com/doc722551386_632783806?hash=gjIfCA0ILqZ1LQlzftCyxZ4zOATANYnUqZXiZ1vsAJH&dl=5wFKrFiIzvVfYJ6M4m1z9ALqKzGdXJdsGAXv1NaBtSg
https://vk.com/doc722551386_632705478?hash=mXInLmfkZNSLz5UVqRoRW60bRlzynUFUpRZoiBeW4ko&dl=zFzHm0Edhycg4ulJp33jdeFbypSaynNcjpZ41cUnID0
...
https://vk.com/doc623586997_607921843?hash=c6f706ee5f09f4d4e5&dl=f780520e509b9f671b
https://vk.com/doc623586997_607809766?hash=ef486a0fb1e873640e&dl=eeb60781cef9e58541

Here are some related questions:

user_183210420
Related