I'm trying to run Firefox with Seleniumwire on an AWS EC2 instance. Here's my current configuration:
- ubuntu==20.4
- selenium-wire==4.3.0
- python==3.8.10
- geckodriver==0.29.1
- ChromeDriver==94.0.4606.81
- 64-bit (Arm)
I've installed ChromeDriver with sudo apt-get install chromium-chromedriver and geckodriver with sudo apt-get install firefox-geckodriver.
Here's the script I'm trying to run:
from seleniumwire import webdriver
options = webdriver.FirefoxOptions()
options.add_argument("--headless")
options.add_argument('--no-sandbox')
options.add_argument('--ignore-certificate-errors-spki-list')
options.add_argument('--ignore-ssl-errors')
options.add_argument("--window-size=1920,1080")
options.add_argument("--disable-gpu")
d = webdriver.Firefox(options=options)
d.get("https://google.com")
print(d.requests)
The problem: d.requests returns [], which is unexpected behavior. I know there's been issues posted about driver.requests returning an empty list, but the recommended solutions haven't worked on my setup. Also print(d.page_source) works fine, which leads me to think there's some incorrect proxy configuration. Also, I isolated this problem to Firefox, since the same code works on Chrome (with Firefox swapped out for Chrome); that is, d.requests contains a non-empty list of requests. I would use Chrome, but the scope of my project requires Firefox.
Here're some steps I've taken to debug:
- Setting the port explicitly using
selenium_options={"port": 4444}, where4444is taken from the localhost output ofgeckodriveron the command line. - Adding a proxy using the following format:
options = {
'proxy': {
'http': 'http://myusername:password@myproxyserver.com:123456',
'https': 'http://myusername:password@myproxyserver.com:123456',
'no_proxy': 'localhost,127.0.0.1'
}
}
- Re-installing and re-running all my scripts on a new EC2 instance.
- Specifying a different
tmpdirectory, per this issue
I have yet to try different versions of geckodriver and selenium-wire, but I suspect that's not the issue.
Any suggestions on debugging would be greatly appreciated.