I have been using scrapy to collect information from the web for some time now, however recently, I started working on a project that required me to collect data from this webpage. Now, before I start collecting anything, I like to test my selectors on the scrapy shell so that I am sure they work.
However, when I tried downloading the webpage https://www.sports-reference.com/cfb/boxscores/index.cgi?month=9&day=12&year=2022&conf_id on scrapy shell, I realised it was not downloading it, I tried changing the URL to this https://www.sports-reference.com/cfb/boxscores/ and it worked just fine.
I am confused, I enclosed the URLs inside double quotes, and it still made no difference.
I got this error:
'day' is not recognized as an internal or external command,
operable program or batch file.
'year' is not recognized as an internal or external command,
operable program or batch file.
'conf_id' is not recognized as an internal or external command,
operable program or batch file.