I am trying to use boto3 to launch a glue development endpoint with the following code:
import boto3
glueClient = boto3.client('glue', 'us-east-1')
glueClient.create_dev_endpoint(
EndpointName = name,
RoleArn = role,
PublicKey = publicKey,
NumberOfNodes = 2,
GlueVersion= "1.0",
Arguments = {
"GLUE_PYTHON_VERSION": "3"
}
)
However I am getting the error:
Parameter validation failed:
Unknown parameter in input: "GlueVersion", must be one of:
EndpointName, RoleArn, SecurityGroupIds, SubnetId, PublicKey,
PublicKeys, NumberOfNodes, ExtraPythonLibsS3Path, ExtraJarsS3Path,
SecurityConfiguration, Tags, Arguments
Which I understand; It's saying I can't specify the glue version in this way. However, boto3's docs state otherwise and claim that GlueVersion is a valid parameter. What gives? I need spark 2.4 and this way it defaults to Spark 2.2. I've tried adding the GlueVersion parameter in the Arguments dict to no avail
EDIT: I should add I am using the latest version of Boto3; 1.14.5