Ruby: Concurrent::Semaphore yields deadlock

user_5673864 0
Viewed 229

I'm testing Concurrent::Semaphore,

require 'concurrent'

loop do
  semaphore = Concurrent::Semaphore.new(3)

  (1..5).each_with_object([]) do |_n, result|
    result << Thread.new do
      semaphore.acquire

      print '.'

      semaphore.release
    end
  end.each(&:join)
end

but the code yields exception like below in several seconds to run.

./semaphore.rb:14:in `join': No live threads left. Deadlock? (fatal)
2 threads, 2 sleeps current:0x00007f867c9a73a0 main thread:0x00007f867a503350
* #<Thread:0x00007f867a869c20 sleep_forever>
   rb_thread_t:0x00007f867a503350 native:0x00007fffa9b16380 int:0
   ./semaphore.rb:14:in `join'
   ./semaphore.rb:14:in `each'
   ./semaphore.rb:14:in `block in <main>'
   ./semaphore.rb:3:in `loop'
   ./semaphore.rb:3:in `<main>'
* #<Thread:0x00007f867b8891f0@./semaphore.rb:7 sleep_forever>
   rb_thread_t:0x00007f867c9a73a0 native:0x000070000c9b8000 int:0
    depended by: tb_thread_id:0x00007f867a503350
   ./semaphore.rb:10:in `write'
   ./semaphore.rb:10:in `print'
   ./semaphore.rb:10:in `block (3 levels) in <main>'

Is something wrong of the usage?

(with MRI ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-darwin17])

deadlock ruby semaphore
1 Answers

The (apparent) deadlock is not directly caused by your usage of semaphores. Instead, what is happening here is that you have two threads (both of which you are waiting to finish) which are both blocking.

Your first thread is indeed waiting for the semaphore to be available.

However the second thread is currently writing data to STDOUT which in your case is also blocking. This can usually happen if the process reading the STDOUT of your Ruby process (e.g. your terminal) is not quick enough to read all of the data. Once the pipe's buffer is full, writing to STDOUT blocks, resulting in the thread to not be live either.

This is detected by Thread#join resulting in the exception being thrown.

To resolve this issue, you could just make sure that you are reading fast enough from your process's STDOUT. Then, I could not reproduce the issue anymore.

For documentation purposes: I could consistently reproduce the issue described by OP by running ruby ./semaphore.rb | ruby -e "sleep 30" with semaphore.rb containing the code show in the question.

user_4217050
Related