Scrapy custom settings

user_1231450 0
Viewed 6089

Using scrapy, I have in one of my spiders:

class IndexSpider(scrapy.Spider):
    name = "indices"

    def __init__(self, *args, **kwargs):
        super(IndexSpider, self).__init__(*args, **kwargs)

        # set custom settings
        custom_settings = {
            'DOWNLOAD_DELAY': 2,
            'ITEM_PIPELINES': {
                'freedom.pipelines.IndexPipeline': 300
            }
        }

However, when I later try to access the settings via

    print(dict(self.settings.get('ITEM_PIPELINES')))

they are empty. Background is that I want to control the settings (and possible pipelines) on a per-spider basis.
What am I doing wrong here?

scrapy python
1 Answers

custom_settings is supposed to be a class attribute:

class IndexSpider(scrapy.Spider):
    name = "indices"

    # set custom settings
    custom_settings = {
        'DOWNLOAD_DELAY': 2,
        'ITEM_PIPELINES': {
            'freedom.pipelines.IndexPipeline': 300
        }
    }

    def __init__(self, *args, **kwargs):
        super(IndexSpider, self).__init__(*args, **kwargs)
user_10846470
Related