There is a web page where I can download zip files using "Save Link As" option in chrome but when I copy link address and open that in the browser it returns 403/forbidden. I tried to save the file using requests library but it also gets forbidden response.
I don't know how chrome can download it but I can't download using requests library.
How can I download the file without using selenium web driver as that will be overkill for this simple task?
I would recommend using requests for this. Simple example below with the first file filled in:
url = 'https://www.nseindia.com/content/historical/EQUITIES/2003/DEC/cm01DEC2003bhav.csv.zip'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.93 Safari/537.36', 'Referer': 'https://www.nseindia.com/'}
r = requests.get(url, allow_redirects=True, headers=headers)
open('cm01DEC2003bhav.csv.zip', 'wb').write(r.content)
The website checks for referer in the header, if referer doesn't match with the website itself it denies the request.
Use urllib.request.urlretrieve with a custom Referer header like @Douglas specified:
>>> import urllib.request
>>> opener = urllib.request.build_opener()
>>> opener.addheaders = [('Referer', 'https://www.nseindia.com/')]
>>> urllib.request.install_opener(opener)
>>> source = 'https://www.nseindia.com/content/historical/EQUITIES/2001/JAN/cm01JAN2001bhav.csv.zip'
>>> destination = 'destination.csv.zip' # Path to destination.
>>> urllib.request.urlretrieve(source, destination)
('destination.csv.zip', <http.client.HTTPMessage object at 0x10ce20208>)
This will download your file to the specified file path.
