AWS Glue:NotADirectoryError: [Errno 20] Not a directory: '/tmp/glue-python-scripts-XXXXX/.YYYY' -> '/tmp/glue-python-scripts-XXXX/'

Hi I am trying to run an AWS Glue job with --TempDir argument:

arguments = {
        "Name": "pythonshell",
        "PythonVersion": "3",
        "ScriptLocation": f"{script_location}",
        "--extra-py-files": path_to_whl,
        "--TempDir": f"s3://{artifacts_bucket_name}/etl-scripts/"
    }
response = glue.start_job_run(JobName=gluejobname, Arguments=arguments)

but I am getting the following Error.

Traceback (most recent call last):  File "/tmp/runscript.py", line 114, in <module>    temp_file_path = download_user_script(args.scriptLocation)  File "/tmp/runscript.py", line 91, in download_user_script    download_from_s3(args.scriptLocation, temp_file_path)  File "/tmp/runscript.py", line 81, in download_from_s3    s3.download_file(bucket_name, s3_key, new_file_path)  File "/usr/local/lib/python3.6/site-packages/boto3/s3/inject.py", line 172, in download_file    extra_args=ExtraArgs, callback=Callback)  File "/usr/local/lib/python3.6/site-packages/boto3/s3/transfer.py", line 307, in download_file    future.result()  File "/usr/local/lib/python3.6/site-packages/s3transfer/futures.py", line 106, in result    return self._coordinator.result()  File "/usr/local/lib/python3.6/site-packages/s3transfer/futures.py", line 265, in result    raise self._exception  File "/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py", line 126, in __call__    return self._execute_main(kwargs)  File "/usr/local/lib/python3.6/site-packages/s3transfer/tasks.py", line 150, in _execute_main    return_value = self._main(**kwargs)  File "/usr/local/lib/python3.6/site-packages/s3transfer/download.py", line 603, in _main    osutil.rename_file(fileobj.name, final_filename)  File "/usr/local/lib/python3.6/site-packages/s3transfer/utils.py", line 271, in rename_file    rename_file(current_filename, new_filename)NotADirectoryError: [Errno 20] Not a directory: '/tmp/glue-python-scripts-tosi3ba1/.f2AaeCED' -> '/tmp/glue-python-scripts-tosi3ba1/'

Script location and path_to_whl exist however there seem to be a problem with creating the tmp directory. Does anyone have an idea why this is happening? The problem occurs when i create the job from CloudFormation stack (the actual script location and python library path are only passed during job run). When I tried creating the job from the console and put the actual script location and python library path during job creating then problem is solved. I don't know why this is happening.