Extend max connections - Python websockets 9.1 and simple-websocket-server 0.4.1

Viewed 568

I wish I could have about 3k connections at once, but when running the server I'm having a bad crash (on about 500 connections in Win 10 x64 and on about 1019 connections on Ubuntu) with both of websockets libraries:

Here is the stresstestscript.py which cause the issue (even if I run this one in different machines over the lan):

import websocket,time
serveraddr="ws://localhost:12345"
ws = websocket.WebSocket()
ws.connect(serveraddr)
list = []
for b in range(3000):
    list.append(websocket.WebSocket())
count=1
for x in  list:
    count+=1
    x.connect(serveraddr)
    time.sleep(0.01)
time.sleep(5000)

Here is the server.py (the exact sample on websockets official documentation):

#!/usr/bin/env python

# WS server example that synchronizes state across clients

import asyncio
import json
import logging
import websockets

logging.basicConfig()

STATE = {"value": 0}

USERS = set()


def state_event():
    return json.dumps({"type": "state", **STATE})


def users_event():
    return json.dumps({"type": "users", "count": len(USERS)})


async def notify_state():
    if USERS:  # asyncio.wait doesn't accept an empty list
        message = state_event()
        await asyncio.wait([user.send(message) for user in USERS])


async def notify_users():
    if USERS:  # asyncio.wait doesn't accept an empty list
        message = users_event()
        await asyncio.wait([user.send(message) for user in USERS])


async def register(websocket):
    USERS.add(websocket)
    print("add")
    await notify_users()


async def unregister(websocket):
    USERS.remove(websocket)
    await notify_users()


async def counter(websocket, path):
    # register(websocket) sends user_event() to websocket
    await register(websocket)
    try:
        await websocket.send(state_event())
        async for message in websocket:
            data = json.loads(message)
            if data["action"] == "minus":
                STATE["value"] -= 1
                await notify_state()
            elif data["action"] == "plus":
                STATE["value"] += 1
                await notify_state()
            else:
                logging.error("unsupported event: %s", data)
    finally:
        await unregister(websocket)


start_server = websockets.serve(counter, "localhost", 12345)

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

Here is the server error (on windows, when exceed the 500 connections opened at once):

Traceback (most recent call last):
  File "websockettestserver.py", line 70, in <module>
    asyncio.get_event_loop().run_forever()
  File "C:\Python37-32\lib\asyncio\base_events.py", line 539, in run_forever
    self._run_once()
  File "C:\Python37-32\lib\asyncio\base_events.py", line 1739, in _run_once
    event_list = self._selector.select(timeout)
  File "C:\Python37-32\lib\selectors.py", line 323, in select
    r, w, _ = self._select(self._readers, self._writers, [], timeout)
  File "C:\Python37-32\lib\selectors.py", line 314, in _select
    r, w, x = select.select(r, w, w, timeout)
ValueError: too many file descriptors in select()

Like I said, I wish I could have more simultaneous connections, increasing the about 1000 connections on Linux to 3000-4000 (!?). How to fix this?

1 Answers

1.run more diff server.py on diff port
2.use nginx to Distribute the requests

Related