We are using websockets to connect to devices. I am writing a websocket client in C# and connecting to a large number of devices (100+) over slow networks. The devices implement a websocket server. My client program will connect to a device, authorise and then request some data from the device - the answer could be several hundred megabytes. And since the devices are on slow networks, it could take minutes. The response should be saved to disk for later processing. And I need to stream this so I don't run out of memory, since I am likely to be downloading from several devices at the same time.
The problem is that all the websocket client libraries I have found only calls you back when an entire message has been received. For example, in websocket-sharp you write
ws.OnMessage += (sender, e) => {
... do something with e.RawData
};
Other client libraries seem to be the same.
But in my case, e.RawData will be 100+ Mb. And I might be receiving 10 of these at the same time, so I must flush data to disk as it arrives, not waiting for a complete message.
Do I have to write my own websocket client implementation, or are there other options? Changing the protocol to send the data in several smaller chunks is not an option.