I currently finalising a Scrapy project however I have quite a lengthy pipelines.py file.
I noticed that in my settings.py the pipe lines are show as follows (trimmed down):
ITEM_PIPELINES = {
'proj.pipelines.MutatorPipeline': 200,
'proj.pipelines.CalculatorPipeline': 300,
'proj.pipelines.SaveToFilePipeline': 500,
}
I have tried the following ways to rectify this.
1.) I created a new file/folder and tried referencing it from pipeline in the same manner.
Folder was myPipelines/Test.py with a class name TestPipeline then referenced in pipeline settings as proj.myPipelines.Test.TestPipeline': 100,.
This threw me errors.
I then thought I could export the module and import into my current pipelines.py and it would take the reference from that. I added an empty __init__.py in my myPipelines directory and then added from myPipelines.Test import TestPipeline but scrapy still throws an error of...
Raise NameError("Module '%s' doesn't define any object named '%s'" % (module, name))
exceptions.NameError: Module 'proj.pipelines' doesn't define any object named 'TestPipeline'.
Many thanks in advance!