The set_extra_http_headers method doesn't work

Viewed 29

I am new to Playwright and trying to create a Scrapy Middleware that uses Playwright to make a request and returns a response.body.

The problem is that I am trying to send headers with my request, but both these methods don't work for me:

browser_context.set_extra_http_headers(headers)

or

page.set_extra_http_headers(headers)

My code looks like this:

async def navigate_page(self, url, headers):
    async with async_playwright() as p:
        browser = await p.chromium.launch()
        context = await browser.new_context()
        # At first I tried this method, but it didn't work, so I decided to set headers with page method
        # context.set_extra_http_headers(headers)
        page = await context.new_page()
        await page.set_extra_http_headers(headers)
        await page.goto(url)
        print(page.request.headers) # showed nothing, debug also showed empty headers
        page_content = await page.content()
        await browser.close()
        return page_content

My headers variable looks like this:

{'referer': 'some url'}

I am pretty sure that I missed something but I searched every article that Google could find and came here in desperation :)

0 Answers
Related