I’m working on a project involving websockets using python-socketio. Our primary concern is fluidity, each connected user will have a cursor whose position on the board is sent as event every 50ms, boards are identified as (socket) rooms, and we are expecting many of these.
I’m new to PubSub, we are horizontally scaling our architecture and it seems to be the fit for events broadcasting.
I had a look at AsyncRedisManager class and from my understanding, it seems any message, sent by any socket on any socketio server (with pub/sub) is then transmitted / published from this server to redis on a single channel of communication. Subscribers to this channel can then see this flow of messages.
I’m hence concerned about 3 things:
- Since all messages are simply going through one channel, isn’t this a “design flaw” as some servers might have no sockets connected to “one” specific room (at the moment), still they will be receiving (and pickle.loading), messages they don’t care about at that time.
- The actual details of these messages (room, payload, etc.) is pickled.dumped and pickle.loaded by servers. In case of 50 rooms with 50 cursors each sending 25 event/s, isn’t this gonna be a huge CPU-bound bottleneck ?
- I’m wrapping my head around the socket.io docs, comparing side by side the redis adapter to Python-socketio pubsub manager, and it seems channels are dynamically namespaced like “socketio#room_name” and messages broadcasted to these “namespaced” channels so that psubscribe would be a viable solution. Some other MQ refer in the terms of “topics”.
- if the former assumption, is correct, still we cannot assume whether one server should or should not psubscribe to a channel#room_name unless no or at least one socket for that server is in the room.
I understand the paradigm of pub/sub is, from Redis page:
Rather, published messages are characterized into channels, without knowledge of what (if any) subscribers there may be. Subscribers express interest in one or more channels, and only receive messages that are of interest, without knowledge of what (if any) publishers there are.
But my question would be summarized as: is it possible to make Python-socketio servers dynamically subscribe/unsubscribe to channels whenever there is a need for it, with channels identified as rooms, hence having as many channels as rooms in total. Would that be feasible while keeping this “plug-&-play” simple logic as a pubsubManager subclass? Am I missing something or does this make sense ?
Thank you for your time, any ideas, corrections, or “draft” code would be greatly appreciated.