I have been given a python script which grabs data from various webpages. My task is to wrap this script in a Talend job and publish it to Talend Cloud where it will be deployed to Remote Engine.
I have built several similar jobs (at least with regard to Talend running python code) in the past. In all cases the python environment was created using Conda and supplied environment.yml, requirements.txt files. I believe there are no issues with the interpreter as I am able to run the script directly on the machine where the Remote Engine is installed, both from Talend studio and from command line.
The problems occur after deploying the job to the Remote Engine via Talend Cloud - It appears Selenium cannot locate Chrome?
From Studio and Command line the console prints:
====== WebDriver manager ====== Current google-chrome version is 105.0.5195 Get LATEST chromedriver version for 105.0.5195 google-chrome Driver [C:\Users\User\.wdm\drivers\chromedriver\win32\105.0.5195.52\chromedriver.exe] found in cache
From Remote Engine Logs:
====== WebDriver manager ====== Could not get version for google-chrome.Is google-chrome installed? Get LATEST chromedriver version for None google-chrome Driver [C:\WINDOWS\system32\config\systemprofile\.wdm\drivers\chromedriver\win32\105.0.5195.52\chromedriver.exe] found in cache
In addition to above, the Remote Engine also shows traceback which reads:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot create temp dir for user data dir
I am stumped. I am hoping the errors above point to some obvious permission/user configuration issue that someone can recognize.
I am running: Windows 10.0.19043, Anaconda 2021.05, Python 3.9, Talend Remote Engine 2.10.5-202
Thank you in advance.