Mongodb connection url not working in AWS Glue Connection screen

Viewed 505

I am trying to connect to mongodb from AWS glue connection screen and getting error

tried various url's

mongodb://server1:2066,server2:2066,server3:2066/dbname.collection?authMechanism=PLAIN&authSource=&external

mongodb://server1:2066,server2:2066,server3:2066/?authMechanism=PLAIN&authSource=&external

Error in AWS Glue connection output in cloud watch

Check that your connection definition references your Mongo database with correct URL syntax, username, and password.
Exiting with error code 30

It works in Glue job with same VPC/Subnet/Security groups

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext, SparkConf
from awsglue.context import GlueContext
from awsglue.job import Job
import time
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

## @params: [JOB_NAME]
args = getResolvedOptions(sys.argv, ['JOB_NAME'])

sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session

job = Job(glueContext)
job.init(args['JOB_NAME'], args)


mongo_uri = "mongodb://_SERVER_1:2066,_SERVER_2:2066,_SERVER_3:2066/?authMechanism=PLAIN&authSource=$external"

read_mongo_options = {
    "uri": mongo_uri,
    "database": "_DB_",
    #"authMechanism": "PLAIN",
    #"authSource": "$external",
    "collection": "_collection_",
    "username": "****",
    "password": "***",
    "partitioner": "MongoSamplePartitioner",
    "partitionerOptions.partitionSizeMB": "10",
    "partitionerOptions.partitionKey": "_id",
    "inferSchema": "false"
}
    
logger = glueContext.get_logger()
logger.info("Connecting...")
s3_location = 's3://_bucket_/output/'


dynamic_frame = glueContext.create_dynamic_frame.from_options(connection_type="mongodb",
                                                            connection_options=read_mongo_options)
print("Connected!")
0 Answers
Related