Error while printing variables from item container in pipeline, python scrapy

user_11894930 0
Viewed 656

I am unable print scraped value from pipeline using python scrapy.

in my python scrapy code i am yielding values to items container and these items values need to be store in pipeline and return value. But it was throwing me type errors. Could you please help me here.

pipeline.py

class JnjwcpocPipeline(object):

    def process_item(self, item, spider):

        print("Pipeline :" + item['product_item'][0])
        return item

Pipeline : JNJ BABY SOAP

ERROR: Error processing {'product_item': None, 'wprice_item': None} Traceback (most recent call last):

current.result = callback(current.result, *args, **kw)

print("Pipeline :" + item['product_item'][0])

TypeError: 'NoneType' object is not subscriptable

scrapy python
3 Answers

item['product_item'] seems to be None,so item['product_item'][0] is not callable, try to make it not None but a container and test it out.

If that's the problem, you can probably use some if to check it if it's None

user_115164190

If you’re trying to print items that might be ‘None’ try enclosing the print function in a ‘try:except’ block that will catch any exceptions and allow the items that aren’t None to be returned:

class JnjwcpocPipeline(object):

def process_item(self, item, spider):
    try:
        print("Pipeline :" + item['product_item'][0])
        return item
    except:
        print('This has a NoneType object.')

Or:

class JnjwcpocPipeline(object):

def process_item(self, item, spider):
    try:
        print("Pipeline :" + item['product_item'][0])

    except:
        pass

    finally:
        return item
user_56869440

The error you are getting is telling you that item['product_item'] is None type and therefore is not subscriptable. You should be checking this before trying to do: item['product_item'][0].

class JnjwcpocPipeline(object):

    def process_item(self, item, spider):
        product_item = item['product_item']
        if product_item is not None: # <-- Check here
            print("Pipeline :" + product_item[0])
        return item
user_20896750
Related