How to improve and load test performance of a flask socketio server? Threading or multiprocessing?

Viewed 206

I have a simple flask socketio server that continuously takes in streams of audio input from a client for every 2.5ms. The flask server passes that stream of audio to an ML model which detects the audio and gives us a result.

My code looks like this

from argparse import Namespace
from flask import Flask
from flask_socketio import SocketIO, emit
from flask_cors import CORS, cross_origin
import detector

app = Flask(__name__)
cors = CORS(app)
app.config['CORS_HEADERS'] = 'Content-Type'
socketio = SocketIO(app, cors_allowed_origins="*")

@cross_origin()
@socketio.on('stream')
def test_message(message):
    result = detector.detect(message['data']) # process which can take some time to run
    return result

if __name__ == '__main__':
    socketio.run(app,host='0.0.0.0')

Now let's assume there are 1000 clients connected to this and they are streaming around 80k-100k bytes of 2.5 ms audio to the server. How do I make sure my server can handle this? Do I need to implement threading or multiprocessing? Do I need to run the socket for each client in a separate thread? How do I load test this server?

0 Answers
Related