I am trying to download urls from https://timesofindia.indiatimes.com/india. However getting error.
This is the code I'm trying:-
urllist=[]
url=requests.get("https://timesofindia.indiatimes.com/india")
content=url.content
soup=BeautifulSoup(content,'lxml')
counter=0
for divtag in soup.find_all('div',{'class':'container wrapper clearfix'}):
for ultag in divtag.find_all('ul',{'class':'list5 clearfix'}):
if (counter<=30) :
for litag in divtag.find_all('li'):
counter=counter+1
newurl='https://timesofindia.indiatimes.com/india'+litag.find('a')['href']
urllist.append(newurl)
This is the error I'm getting:-
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-67-a0e2d46fa0d4> in <module>
10 for litag in divtag.find_all('li'):
11 counter=counter+1
---> 12 newurl='https://timesofindia.indiatimes.com/india'+litag.find('a')['href']
13 urllist.append(newurl)
TypeError: 'NoneType' object is not subscriptable
Could anyone advise how to correct it?