I am using a stock broker's API to create an application in python. The API has a websocket. I need pass a list of tokens (of stock tickers) and the callback function is called as and when there is a change in any data for those tokens (for example price of the stock). The structure is -
api.start_websocket(order_update_callback=event_handler_order_update, subscribe_callback=event_handler_quote_update, socket_open_callback=open_callback)
In the socket open, I need to subscribe list of tokens. For example:
api.subscribe(['NSE|22', 'BSE|522032'])
I can use event_handler_quote_update to process the streaming data. Such as showing the live prices, Open Interest, Volume etc. on a web page or on a spreadsheet.
The processing of the data will be slower than the speed of streaming data. In that case what should be the strategy or technology to make sure that all the data get processed in real time? Can I use Queue? But it appears that the blocking mechanism may hamper the functionality in this case. Is threading a good solution? If yes, how it should be implemented in this case? Any other options?