I am trying to implement a gateway design to access/abstract the api to my database, which is simply a single HDB and RDB on the same server. Reading through documentation https://code.kx.com/q/wp/gateway-design/ the most basic gateways act as at least one man in the middle. Without aggregation this doubles the data transfer needed and with aggregation it seems that to be generic (implement "select" for example) it would need to pull all data to the gateway anyway (for example to perform an average that required data from both). Is there something I am missing in the design of the gateway so as not to copy the data through it, a simple and elegant solution for a small setup would be ideal. I guess this is the map reduce problem in general but in a KDB+ HDB/RDB setting.
2 Answers
The gateway in general doesn't hold the data. Instead it uses Inter-process Communication (IPC) to send requests to the RDB and HDB. The queries are calculated on the RDB and HDB side and the result is then sent to the gateway which is then sent to the client, but the gateway usually doesn't store the data.
You might want to take a look on https://www.aquaq.co.uk/kdb-gateways/ aquaq torq has implement a good gw setup. If you are setting a new kdb project, torq will be a boost starter for you.
Asyn gw dosent hold any data Sync gw does hold data and processing can be done on sync gw itself and then return to user/client
Related