Issue with Login Authentication on StockX Using Scrapy

user_14025987 0
Viewed 407

Can someone help me with using scrapy to login to StockX.com? I am trying to build a web scraper to scrape price information off of stockx but I am unfamiliar with the type of authentication that Stockx requires to login. I have watched tutorials on using csrf tokens but do not see any tokens submitted in the form data when I login, so I am at a loss at how to authenticate logins using scrapy on StockX. Any help would be much appreciated.

Current code:

class StockXSpider(scrapy.Spider):
    name = 'sx'
    # page_number = 2
    start_urls = [
        'https://www.stockx.com/login? 
 iss=https%3A%2F%2Faccounts.stockx.com%2F'
    ]

def parse(self, response):
    return scrapy.FormRequest.from_response(response,
                                            formdata={"username": "Gabiospi321@gmail.com", "password": "LitMari757"},
                                            callback=self.scrape)

def scrape(self, response):
     items = SxscrapyItem()

    all_div_blanks = response.css('div.product-header-media')

    for blanks in all_div_blanks:
        product_name = response.css('div.col-md-12').xpath('//h1/text()').extract()
        lowest_ask = response.css('div.en-us.stat-value.stat-small::text')[0].extract()
        highest_bid = response.css('div.en-us.stat-value.stat-small::text')[1].extract()

        items['product_name'] = product_name
        items['lowest_ask'] = lowest_ask
        items['highest_bid'] = highest_bid

        yield items
scrapy python
1 Answers

You could use selenium instead and do the following to log into stockx:

Code:

Using selenium you would login by finding the email input and password input and then sending the keys for your login details. I am not sure if stockx have an api but if they did it might be easier to use that instead of logging in and scraping manually.

from selenium import webdriver
import time

driver = webdriver.Firefox() # you could use chrome instead

driver.get("http://accounts.stockx.com/login")
time.sleep(4)  # small delay before inputting login for page to load

driver.find_element_by_id("email-login").send_keys(
    "myemail@email.com"
)  # inputs your email

driver.find_element_by_id("password-login").send_keys(
    "mypassword"
)  # inputs your password

driver.find_element_by_id("btn-login").click() # clicks the login button

Video Demo:

Video

user_137762470
Related