error: can't start new thread

user_190127 0
Viewed 115853

I have a site that runs with follow configuration:

Django + mod-wsgi + apache

In one of user's request, I send another HTTP request to another service, and solve this by httplib library of python.

But sometimes this service don't get answer too long, and timeout for httplib doesn't work. So I creating thread, in this thread I send request to service, and join it after 20 sec (20 sec - is a timeout of request). This is how it works:

class HttpGetTimeOut(threading.Thread):
    def __init__(self,**kwargs):
        self.config = kwargs
        self.resp_data = None
        self.exception = None
        super(HttpGetTimeOut,self).__init__()
    def run(self):

        h = httplib.HTTPSConnection(self.config['server'])
        h.connect()
        sended_data = self.config['sended_data']
        h.putrequest("POST", self.config['path'])
        h.putheader("Content-Length", str(len(sended_data)))
        h.putheader("Content-Type", 'text/xml; charset="utf-8"')
        if 'base_auth' in self.config:
            base64string = base64.encodestring('%s:%s' % self.config['base_auth'])[:-1]
            h.putheader("Authorization", "Basic %s" % base64string)
        h.endheaders()

        try:
            h.send(sended_data)
            self.resp_data = h.getresponse()
        except httplib.HTTPException,e:
            self.exception = e
        except Exception,e:
            self.exception = e

something like this...

And use it by this function:

getting = HttpGetTimeOut(**req_config)
getting.start()
getting.join(COOPERATION_TIMEOUT)
if getting.isAlive(): #maybe need some block
    getting._Thread__stop()
    raise ValueError('Timeout')
else:
    if getting.resp_data:
        r = getting.resp_data
    else:
        if getting.exception:
            raise ValueError('REquest Exception')
        else:
            raise ValueError('Undefined exception')

And all works fine, but sometime I start catching this exception:

error: can't start new thread

at the line of starting new thread:

getting.start()

and the next and the final line of traceback is

File "/usr/lib/python2.5/threading.py", line 440, in start
    _start_new_thread(self.__bootstrap, ())

And the answer is: What's happen?

Thank's for all, and sorry for my pure English. :)

httplib mod-wsgi multithreading django python
9 Answers

I was running on a similar situation, but my process needed a lot of threads running to take care of a lot of connections.

I counted the number of threads with the command:

ps -fLu user | wc -l

It displayed 4098.

I switched to the user and looked to system limits:

sudo -u myuser -s /bin/bash

ulimit -u

Got 4096 as response.

So, I edited /etc/security/limits.d/30-myuser.conf and added the lines:

myuser hard nproc 16384

myuser soft nproc 16384

Restarted the service and now it's running with 7017 threads.

Ps. I have a 32 cores server and I'm handling 18k simultaneous connections with this configuration.

user_64702720

I'm running a python script on my machine only to copy and convert some files from one format to another, I want to maximize the number of running threads to finish as quickly as possible.

Note: It is not a good workaround from an architecture perspective If you aren't using it for a quick script on a specific machine.

In my case, I checked the max number of running threads that my machine can run before I got the error, It was 150

I added this code before starting a new thread. which checks if the max limit of running threads is reached then the app will wait until some of the running threads finish, then it will start new threads

while threading.active_count()>150 :
    time.sleep(5)
mythread.start()
user_3485890

If you are using a ThreadPoolExecutor, the problem may be that your max_workers is higher than the threads allowed by your OS.

It seems that the executor keeps the information of the last executed threads in the process table, even if the threads are already done. This means that when your application has been running for a long time, eventually it will register in the process table as many threads as ThreadPoolExecutor.max_workers

user_33088400

As far as I can tell it's not a python problem. Your system somehow cannot create another thread (I had the same problem and couldn't start htop on another cli via ssh).

The answer of Fernando Ulisses dos Santos is really good. I just want to add, that there are other tools limiting the number of processes and memory usage "from the outside". It's pretty common for virtual servers. Starting point is the interface of your vendor or you might have luck finding some information in files like /proc/user_beancounters

user_137280560
Related