scrapy setting option CONCURRENT_REQUESTS is not working

Viewed 479

Python 3.6.2, Scrapy 1.4.0, windows 2007

scrapy is working fine, but this option CONCURRENT_REQUESTS is not working.

i set CONCURRENT_REQUESTS to 64, but the speed didn't change. cpu rate stays at 20%, this option seems to change nothing.

how can i detect it's actual concurrent_requests and make it run faster

0 Answers
Related