Scrapy - Create Additional Items In Pipeline

Viewed 357

So I currently have 2 pipelines, both are ordered as below.

ITEM_PIPELINES = {
    'myproject.pipelines.mutatorPipeline': 300,
    'myproject.pipelines.submitDatabasePipeline': 800,
}

They execute in the correct order.

1.) The first pipeline is a mutator.

2.) The second is submits the information to the database.

The data correctly gets submitted to the database.

My problems is that sometimes when data reaches my mutator pipeline I want to create 'additional' items to pass to my second pipeline.

Currently (without mutations) it looks like this:

def process_item(self, item, spider):
            #Mutate the item
            return item

I cannot however, return more than once. I also do not want to create an additional item that starts from the start of pipeline 1.

Many helps in advance.

1 Answers
Related