AWS Neptune Connection Failed - Loading Data Into Neptune Cluster from S3 Bucket using CURL command

user_12555986 0
Viewed 547

I am trying to bulk load RDF N-triples data from S3 bucket into Neptune loader. I have created a S3 bucket, IAM Role, Endpoint and Neptune cluster as per the following link https://docs.aws.amazon.com/neptune/latest/userguide/bulk-load-data.html.

And I am trying to execute a CURL command locally from windows using command prompt to load the data,

curl -X POST -H "Content-Type: application/json" https://<clusterEndpoint>:<clusterPort>/loader -d "{\"source\":\"s3://<bucketName>\",\"format\":\"ntriples\",\"iamRoleArn\":\"arn:aws:iam::<account-id>:role/<role-name>\",\"region\":\""<region>\",\"failOnError\":\"FALSE\",\"parallelism\":\"MEDIUM\",\"updateSingleCardinalityProperties\":\"FALSE\",\"queueRequest\":\"TRUE\"}"

On executing the above Curl command I am getting the following error,

Failed to connect to <neptuneClusterEndoint> port <portNumber>: Timed out

Also, when I tried to check the cluster status using the command curl http://<neptuneCluster>:<portNumber>/status , I got the same time out error.

I am trying to make a Neptune Load using Curl command without creating an EC2 instance. May I know why I am getting the connection failed error? Is there a way to use curl command to make a Neptune load successfully?

amazon-neptune amazon-web-services curl
1 Answers

The curl command needs to have access to the Neptune VPC. That could be via an EC2 bastion host over an SSH tunnel for example. As you are trying to avoid using EC2 you will need to setup an alternative way to access Neptune such as a load balancer. You could also use a Lambda function so long as the Lambda function has access to the VPC. There are many other ways you could decide to connect but as Neptune does not expose a public IP address you will need to configure a way for your curl command to access that VPC. Also be aware that if you have IAM Authentication enabled on the Neptune cluster that the request will have to be signed using SigV4 credentials.

If you have a Neptune Notebook configured, you can just use the %load command and it can also handle any SigV4 needs for you as well.

user_54420340
Related