I'm playing with RavenDB ETL and are trying to send data to an sql database. With one Ravendb database that is pretty easy to achive. In my case I have many (20++) RavenDB databases and I would like to send all the data to the SQL reporting database. Since all the databases have the same structure the documentId in the different RavenDB can be the same and the value in the documentId column in SQL database will then not be unique. Do anyone have experierence on how to solve this?
2 Answers
You can simply put extra column which will discriminate source database name.
I know each SQL script will contain different definition, but you can automate this task using any client (c#, java, node, etc).
For API/examples go here: https://github.com/ravendb/ravendb/blob/c8c7f7ac0e8276f6ebee9a40ebacc1320e486a0d/test/SlowTests/Server/Documents/ETL/SQL/SqlEtlTests.cs#L39
Optionally you might concat database name to document id - decision is yours.
Just an idea:
Do the 20++ RavenDB databases have different/unique names ?
If yes, you can concatenate the db name to each document created on those databases
This way each document will get a unique id
The document id can be manipulated with onBeforeStore
See: https://ravendb.net/docs/article-page/5.2/Csharp/client-api/session/how-to/subscribe-to-events
So the db name is concatenated to the db is right before storing to disk...
and this way the SQL ETL gets unique document ids
Related