Could anyone tell me why I am getting an error when using Glue to read a very simple exported DynamoDB json file (in S3 bucket) to a new DynamoDB instance. I have looked at the json and it seems fine
Both DynamoDB instances composite key partition key = 'pk' and range key = 'sk' which are strings, its unbelievably simple its just to test it out.
Heres the relevant part of the script
ApplyMapping_node2 = ApplyMapping.apply(
frame=S3bucket_node1,
mappings=[
("Item.pk.S", "string", "Item.pk.S", "string"),
("Item.sk.S", "string", "Item.sk.S", "string") ],
transformation_ctx="ApplyMapping_node2",
)
S3bucket_node3 = glueContext.write_dynamic_frame.from_options(
frame=ApplyMapping_node2,
connection_type="dynamodb", connection_options={"dynamodb.output.tableName": "my-table"}
}
But I always get the error when I run the script
WriteDynamicFrame. The provided key element does not match the schema
But both example databases have the same schema , its kind of driving me insane . I don't know how to debug this or resolve it , it seems so straight forward any advice is appreciated.