I have started using Scrapy recently and I'm trying to use the "XMLFeedSpider" to extract and load the pages that are in a xml page. But the problem is that it is returning an error: "IndexError: list index out of range".
I'm trying to collect and load all product pages that are at this address:
"http://www.example.com/feed.xml"
My spider:
from scrapy.spiders import XMLFeedSpider
class PartySpider(XMLFeedSpider):
name = 'example'
allowed_domains = ['http://www.example.com']
start_urls = [
'http://www.example.com/feed.xml'
]
itertag = 'loc'
def parse_node(self, response, node):
self.logger.info('Hi, this is a <%s> node!: %s', self.itertag,''.join(node.extract()))