Is this Sprite SVG possible to scrape?

user_13726006 0
Viewed 121

Can I scrape this with standard Scrapy or do I need to use Selenium?

The html is:

<td class="example"><sprite-svg name="EXAMPLE2"><svg><use 
xlink:href="/spritemap/1_0_30#sprite-EXAMPLE2"></use></svg></sprite-svg></td>

I need the value "EXAMPLE2" somehow.

The xpath which works in the browser is //td[@class='example']//*[local-name() = 'svg']

When I put it into scrapy I use the following code but am getting XPATH error.

'example' : div.xpath(".//td[@class='example']//*[local-name() = 'svg'] 
()").extract()

Any ideas how to scrape it?

sprite scrapy web-scraping svg python
2 Answers

Looking at the table, each svg sprite is under a class 'rug_X'

Something like

import scrapy


class RaceSpider(scrapy.Spider):
    name = 'race'
    allowed_domains = ['thedogs.com.au']
    start_urls = ['https://www.thedogs.com.au/racing/gawler/2020-07-07/1/the-bunyip-maiden-stake-pr2-division1']
    item = {}
    def parse(self, response):
        row = response.xpath('//tbody/tr')

        dog = a.xpath('.//td[@class="table__cell--tight race-runners__name"]/div/a/text()').get()
        
        number = a.xpath('.//td[@class="table__cell--tight race-runners__box"]/sprite-svg/@name').get() 
      
        cleaned_num = int(number.replace('rug_',''))
        grade = a.xpath('.//td[@class="race-runners__grade"]/text()').get()

        item = {'grade':grade, 'greyhound':dog,'rug':cleaned_num}
        yield item

You could also use item loaders with a custom function to clean up the response you get.

user_125660460

Yes. You can do it with scrapy :

 response.xpath("//td[@class='table__cell--tight race-runners__box']/sprite-svg/@name").getall()

Working scrapy code :

import scrapy

class Test(scrapy.Spider):
    name = 'Test'
    start_urls = [
        'https://www.thedogs.com.au/racing/gawler/2020-07-07/1/the-bunyip-maiden-stake-pr2-division1']

def parse(self, response):
    return {"nameList": response.xpath("//td[@class='table__cell--tight race-runners__box']/sprite-svg/@name").getall()}
user_83234420
Related