Is Orion compatible with AWS DocumentDB

Question

I am trying to connect Orion with AWS DocumentDB but it's not getting connected. However I tried two other FIWARE components IoTAgent and Sth-Comet with DocumentDB and both are working fine.

Same hostname and credential are working for IoTAgent and Sth-Comet. I also checked for the connectivity, which is fine, as IoTAgent and Sth-Comet are in same network. I also checked from a different mongo host in same network and this also worked. Below is the error that I am getting for Orion.

time=2021-02-18T07:03:46.293Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=mongoConnectionPool.cpp[180]:mongoConnect | msg=Database Startup Error (cannot connect to mongo - doing 100 retries with a 1000 millisecond interval)

Is there any possibility that Orion is not compatible with AWS DocumentDB?

Update1:

bash-4.2$ ps ax | grep contextBroker
    1 ?        Ss     0:00 /usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -disableFileLog -dbhost xxxxxxxxxxxxxxxxxx.docdb.amazonaws.com -db admin -dbuser test -dbpwd xxxxxxxxxx

Update2:

Earlier, I was using Orion docker images by pulling directly from dockerhub and that was not working. So this time, I build two docker images by building source code of version 2.4.2 and 2.5.2. Now, I was able to connect with AWS DocuemntDB with these docker images but getting a different error as below.

time=2021-02-23T06:10:41.982Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=safeMongo.cpp[360]:getField | msg=Runtime Error (field '_id' is missing in BSONObj <{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported" }> from caller mongoSubCacheItemInsert:83)
time=2021-02-23T06:10:41.982Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=AlarmManager.cpp[211]:dbError | msg=Raising alarm DatabaseError: error retrieving _id field in doc: '{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported" }'

Below is the Orion version

contextBroker --version
2.5.0-next (git version: 3984f9fc30e90fa04682131ca4516b4d277eb27e)

curl -X GET 'http://localhost:1026/version'
    {
    "orion" : {
      "version" : "2.5.0-next",
      "uptime" : "0 d, 0 h, 4 m, 56 s",
      "git_hash" : "3984f9fc30e90fa04682131ca4516b4d277eb27e",
      "compile_time" : "Mon Feb 22 17:39:30 UTC 2021",
      "compiled_by" : "root",
      "compiled_in" : "4c7575c7c27f",
      "release_date" : "Mon Feb 22 17:39:30 UTC 2021",
      "doc" : "https://fiware-orion.rtfd.io/",
      "libversions": {
         "boost": "1_53",
         "libcurl": "libcurl/7.29.0 NSS/3.53.1 zlib/1.2.7 libidn/1.28 libssh2/1.8.0",
         "libmicrohttpd": "0.9.70",
         "openssl": "1.0.2k",
         "rapidjson": "1.1.0",
         "mongodriver": "legacy-1.1.2"
      }
    }
    }

I am also able to connect to DocumentDB from Orion Pod using Mongo Shell.

mongo --host xxxxxxxxxxxxxxxxxx.docdb.amazonaws.com:27017 --username xxxx --password xxxx
rs0:PRIMARY> show dbs;
rs0:PRIMARY>

I am also able to create entries using below command and it creates a DB and collection in DocumentDB:

curl localhost:1026/v2/entities -s -S --header 'Content-Type: application/json' \
>     -X POST -d @- < {
>   "id": "Room2",
>   "type": "Room",
>   "temperature": {
>     "value": 23,
>     "type": "Number"
>   },
>   "pressure": {
>     "value": 720,
>     "type": "Number"
>   }
> }
> EOF

rs0:PRIMARY> show dbs;
orion  0.000GB

But I am not able to get that data using orion API and after executing this command it getting exited from container with a empty response. I have checked the same with Orion version 2.4.2 and 2.5.2 with DocumentDB 4.0 and 3.6.

[root@orion-docdb-7748fd9c85-gbjz7 /]# curl localhost:1026/v2/entities/Room2 -s -S --header 'Accept: application/json' | python -mjson.tool
curl: (52) Empty reply from server
command terminated with exit code 137

At the end, still getting same error in logs.

time=2021-02-23T06:16:04.564Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=safeMongo.cpp[360]:getField | msg=Runtime Error (field '_id' is missing in BSONObj <{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported" }> from caller mongoSubCacheItemInsert:83)
time=2021-02-23T06:16:04.564Z | lvl=ERROR | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=AlarmManager.cpp[211]:dbError | msg=Raising alarm DatabaseError: error retrieving _id field in doc: '{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported" }'

Update3:

I have added -noCache and deployed again. Below are the commands output and logs for your reference.

Process check:

#ps ax | grep contextBroker
1 ?        Ssl    0:00 /usr/bin/contextBroker -fg -multiservice -ngsiv1Autocast -disableFileLog -dbhost xxxxxxxxxxxxxxxxxx.docdb.amazonaws.com -dbuser xxxxxxxx -dbpwd xxxxxxxx -logLevel DEBUG -noCache

Entries in DB:

rs0:PRIMARY> show dbs
orion  0.000GB
rs0:PRIMARY> use orion
switched to db orion
rs0:PRIMARY> show collections
entities
rs0:PRIMARY> db.entities.find()
{ "_id" : { "id" : "Room2", "type" : "Room", "servicePath" : "/" }, "attrNames" : [ "temperature", "pressure" ], "attrs" : { "temperature" : { "type" : "Number", "creDate" : 1614323032.671698, "modDate" : 1614323032.671698, "value" : 23, "mdNames" : [ ] }, "pressure" : { "type" : "Number", "creDate" : 1614323032.671698, "modDate" : 1614323032.671698, "value" : 720, "mdNames" : [ ] } }, "creDate" : 1614323032.671698, "modDate" : 1614323032.671698, "lastCorrelator" : "c8a73f40-7800-11eb-bd9b-bea9c419835d" }

Orion Pod Logs:

time=2021-02-26T06:46:33.966Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1008]:main | msg=start command line 
time=2021-02-26T06:46:33.966Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1076]:main | msg=Orion Context Broker is running
time=2021-02-26T06:46:34.280Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=MongoGlobal.cpp[243]:mongoInit | msg=Connected to mongo at xxxxxxxxxxxxxxxxxx.docdb.amazonaws.com/orion, as user 'xxxxxxx' (poolsize: 10)
time=2021-02-26T06:46:34.282Z | lvl=INFO | corr=N/A | trans=N/A | from=N/A | srv=N/A | subsrv=N/A | comp=Orion | op=contextBroker.cpp[1202]:main | msg=Startup completed
time=2021-02-26T07:03:24.546Z | lvl=INFO | corr=b7e44e5a-7800-11eb-9531-bea9c419835d | trans=1614321993-966-00000000001 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=logTracing.cpp[79]:logInfoRequestWithoutPayload | msg=Request received: GET /version, response code: 200
time=2021-02-26T07:03:52.672Z | lvl=ERROR | corr=c8a73f40-7800-11eb-bd9b-bea9c419835d | trans=1614321993-966-00000000002 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=safeMongo.cpp[360]:getField | msg=Runtime Error (field '_id' is missing in BSONObj <{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported", operationTime: Timestamp 1614323032|1 }> from caller processContextElement:3493)
time=2021-02-26T07:03:52.672Z | lvl=ERROR | corr=c8a73f40-7800-11eb-bd9b-bea9c419835d | trans=1614321993-966-00000000002 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=AlarmManager.cpp[211]:dbError | msg=Raising alarm DatabaseError: error retrieving _id field in doc: '{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported", operationTime: Timestamp 1614323032|1 }'
time=2021-02-26T07:03:52.782Z | lvl=ERROR | corr=c8a73f40-7800-11eb-bd9b-bea9c419835d | trans=1614321993-966-00000000002 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=AlarmManager.cpp[235]:dbErrorReset | msg=Releasing alarm DatabaseError
time=2021-02-26T07:03:52.790Z | lvl=ERROR | corr=c8a73f40-7800-11eb-bd9b-bea9c419835d | trans=1614321993-966-00000000002 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=safeMongo.cpp[360]:getField | msg=Runtime Error (field '_id' is missing in BSONObj <{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported", operationTime: Timestamp 1614323032|1 }> from caller addTriggeredSubscriptions_noCache:1408)
time=2021-02-26T07:03:52.790Z | lvl=ERROR | corr=c8a73f40-7800-11eb-bd9b-bea9c419835d | trans=1614321993-966-00000000002 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=AlarmManager.cpp[211]:dbError | msg=Raising alarm DatabaseError: error retrieving _id field in doc: '{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported", operationTime: Timestamp 1614323032|1 }'
time=2021-02-26T07:03:52.791Z | lvl=INFO | corr=c8a73f40-7800-11eb-bd9b-bea9c419835d | trans=1614321993-966-00000000002 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=logTracing.cpp[130]:logInfoRequestWithPayload | msg=Request received: POST /v2/entities, request payload (148 bytes): {  "id": "Room2",  "type": "Room",  "temperature": {    "value": 23,    "type": "Number"  },  "pressure": {    "value": 720,    "type": "Number"  }}, response code: 201
time=2021-02-26T07:03:58.479Z | lvl=ERROR | corr=cc1d5934-7800-11eb-a28d-bea9c419835d | trans=1614321993-966-00000000003 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=AlarmManager.cpp[235]:dbErrorReset | msg=Releasing alarm DatabaseError
time=2021-02-26T07:03:58.479Z | lvl=ERROR | corr=cc1d5934-7800-11eb-a28d-bea9c419835d | trans=1614321993-966-00000000003 | from=127.0.0.1 | srv= | subsrv= | comp=Orion | op=safeMongo.cpp[360]:getField | msg=Runtime Error (field '_id' is missing in BSONObj <{ ok: 0.0, code: 303, errmsg: "Legacy opcodes are not supported", operationTime: Timestamp 1614323038|1 }> from caller ContextElementResponse:109)
terminate called after throwing an instance of 'mongo::AssertionException'
  what():  assertion src/mongo/bson/bsonelement.cpp:392

Pod exited and restarted during API call:

curl localhost:1026/v2/entities/Room2 -s -S --header 'Accept: application/json' | python -mjson.tool
command terminated with exit code 137

user_1485926 · Answer

The following message shown in log traces is pretty significant

"Legacy opcodes are not supported"

Although the MongoDB driver used by Orion 2.5.2 and before works with official MongoDB version up to 4.4, probably it is not the case with MongoDB "clones" like AWS DocumentDB.

We are in the process to change the legacy driver used by Orion to a new one. Once this change lands in Orion master branch, I'd suggest to test it (using :latest dockerhub tag). In the meanwhile, as a workaround, I'd suggest to use a official MongoDB database.

EDIT: the process to change the MongoDB driver has finished and Orion is using the new driver since version 3.0.0. I think it would be a good idea to test with this new version and see how it goes. I can help with the test if you provide me with the access information (see here).

Is Orion compatible with AWS DocumentDB

1 Answers