kafka vs rest endpoint in terms of data ownership

user_4021776 0
Viewed 53

We have an application which has UI component written in React and service A which holds transactions for each day. As of today, the UI application invokes a REST API on service A to get transactions data and display on UI. UI layer does not owns any business logic, all business logic is in service A.

Now we have a change upcoming where in we have to fetch data from service B, which holds the inventory details for each store, and this data also needs to be displayed on UI. Service B currently publishes inventory details every 3 hours on a Kafka topic.

So one of the proposed designs is this: Service A consume the inventory details from the Kafka topic (which service B is publishing) and store a local copy in its database, and update it every day and expose a REST API to display the data on UI. In my personal opinion, in this case service A just becomes a pass through of data that service B owns/publishes, we do not have any business logic we need to apply we don't produce this data.

Other design option is we force service B to expose a REST endpoint to expose the data.

I'm a bit confused on what should be the right approach here? looking for some pointers.

apache-kafka rest architecture
1 Answers

The first proposal is definitely not the desired one, because of multiple reasons:

  • Service A becomes a bottleneck for those flows, if it breaks, you won't have data in UI from both services A and B
  • By storing data from service B in the storage of service A, you're going to get more coupling between them (if the message schema of service B changes, you need to sync that with service A's storage)
  • What happens if you add new service (C) in the future? The architecture will definitely not scale if the data goes through service A.

The second proposal is better, because some of the above-mentioned concerns will be avoided. However, on the UI layer you still need to know the endpoint of service B and some specific info related to this service (e.g. what headers it accepts, specific request params, connection timeouts, etc.)

Now if we think from an evolutionary standpoint - adding more services in the future and each of them with specific endpoints, connection details, etc, we can quickly notice that the UI layer will become bloated with specific info on how to connect to those services individually.

One way to solve this is to use the Gateway Aggregation Pattern which a thin layer placed between the UI and the backend services, which mediates the communication between them. This way you get more consistency on the UI (it only needs to communicate to a single endpoint), while keeping the backend services independent.

Of course, this comes with pros and cons, it's not a silver bullet, but seems to fit well on this particular use-case.

user_27873640
Related