Sync data from REST and Websockets

user_6456586 0
Viewed 682

we are building chat application similar to messenger. There is required behavior:

  • User log in
  • User should see last N messages, and he should be able to load older messages
  • New messages should be appended as well

My solution:

  • I would like to use websockets for this purpose with combination of REST. My idea was that client application decide by message id which messages need. So REST will be used for initial fetching of messages and fetching older messages.
  • New messages will received by websockets

Possible issue which I should handle:

  • Application starts subscribing websocket channel for new messages and send request for old messages without initial message id
  • There is chance that after calling GET request new message come, and will be stored in DB
  • Client application started subscribing websocket channel so message will received by websockets.
  • GET request didn't know about this message and fetch last N messages where this new messages will occured and client application will have duplicate record and have to filtered this messages

Can you give me advice if there is some elegant way how to handle this case? Thank you.

software-design rest websocket
1 Answers

I would resolve your task having in mind the following:

The client application should know only about the topic to which to listen. And not the ID of the message starting from which to listen.

It is up to the server to decide what to return (even time should always be tracked server-side).

The WebSocket is used as a transport for STOMP (simply to not reinvent the wheel). The WebSocket connection could be opened once the client application is loaded and not when it is entering the "listen for messages" state. But topic subscription should be performed when necessary.

You can always send GET request and initiate a STOMP subscription simultaneously (almost simultaneously, well with a delay of 1-2 nano-second). And those always should be processed in different promises. But I would align those in the following way: first, the STOMP subscription is initiated, And a specific message on subscription with the initial timestamp of the start of subscription is delivered; second, REST request to get previous 10-100 messages for the TOPIC prior to a specific timestamp (received from STOMP) is performed.

Getting the last 10 messages (which are prior to subscription moment) could be delivered as by REST as by STOMP approach: you can always react to a subscription event on your server-side, and deliver client-specific messages.

Regarding the problem of multiple identical messages from different "data channels", it is easily resolvable: your client (hope that is not jquery, but rather Angular or React or Vue or anything else) will be storing all the data in a single collection in a controller, and filtering and checking by message-id that only unique entries are stored is easy.

BUT if your system will produce hundreds of thousands of messages per second: I guess HTTP-based protocols are not your choice in this case.

user_4509510
Related