Display an item search in ajax format from an ecommerce website

Nail. G 0
Viewed 33

I am trying to scrape an ecommerce website (Lazada.sg) and I found a Github code based of on scrapy: https://github.com/talk2div/lazada-scraper. As I'm tinkering how he developed his code, however, I cannot replicate displaying the same URL search in ajax format (correct me if I'm wrong). Here is a sample of the URL for request in scrapy:https://www.lazada.sg/mother-baby/?ajax=true&page=1&spm=a2o42.searchlistcategory.cate_5b6ee3f0Npltyg.

The searches he made are for baby item searches. I am trying to replicate that for Lego items. I would be glad if I can have some help on displaying the URL in the same format as he did for scrapy so I can re-use his code for my own use case. Thanks

ajax scrapy python
1 Answers

That is because the links he is querrying is part of an enumerated category listed on the page. You just want to get the search results of a specific keyword so the query will look a little different like this:

...
page=1
def start_requests(self):
        yield scrapy.Request(url=f'https://www.lazada.sg/catalog/?_keyori=ss&ajax=true&from=input&isFirstRequest=true&page={self.page}&q=lego&spm=a2o42.searchlistcategory.search.go.d1c332ab2wBQx9')

This is the link for the first page.

https://www.lazada.sg/catalog/?_keyori=ss&ajax=true&from=input&isFirstRequest=true&page=1&q=lego&spm=a2o42.searchlistcategory.search.go.d1c332ab2wBQx9

userphOc10
Related