Coroutines - Dispatchers.Main.immediate with join is deadlocking inside runBlocking

Viewed 1196

Breaking down a simple case on Android to suspend the main thread and perform concurrent processing with coroutines, the following code only prints Launched and runBlocking never completes:

runBlocking {
      val ioJob = launch(Dispatchers.IO) {
        delay(1000)
      }
      val mainJob = launch(Dispatchers.Main.immediate) {
        Log.d("Routine", "Launched")
        ioJob.join()
        Log.d("Routine", "Joined")
      }

      listOf(mainJob, ioJob).joinAll()
    }

Of course if we replace Dispatchers.Main.immediate with Dispatchers.IO everything works great, but some of my concurrent processing should be run on main. Using Dispatchers.Main doesn't log anything, as expected. It appears that once a join is performed within the root of runBlocking it paralyzes anything suspended that was dispatched to main.

Worth noting, an approach predicated on CountDownLatch and threads works great:

    val latchMain = CountDownLatch(1)
    val primaryLatch = CountDownLatch(2)

    val ioExecutor = Executors.newCachedThreadPool(Executors.defaultThreadFactory())

    log("Execute IO")
    ioExecutor.execute(
        Runnable {
          log("Delay IO")
          Thread.sleep(1000)
          log("Countdown Main")
          latchMain.countDown()
          Thread.sleep(3000)
          primaryLatch.countDown()
        })

    log("Execute Main")
    Runnable {
      log("Await Main")
      latchMain.await()
      log("Countdown Primary")
      primaryLatch.countDown()
    }.run()

    log("Await Primary")
    primaryLatch.await()
    log("Continue")

    stepTracker.endTracking()
    return stepTracker.stepGraphTrace
  }

  private fun log(msg: String) = Log.i("Routine", "[${Thread.currentThread().name}] $msg")

With output:

2021-08-11 11:04:06.508 [main] Execute IO
2021-08-11 11:04:06.509 [main] Execute Main
2021-08-11 11:04:06.510 [main] Await Main
2021-08-11 11:04:06.510 [pool-25-thread-1] Delay IO
2021-08-11 11:04:07.512 [pool-25-thread-1] Countdown Main
2021-08-11 11:04:07.513 [main] Countdown Primary
2021-08-11 11:04:07.513 [main] Await Primary
2021-08-11 11:04:10.514 [main] Continue

Any ideas? About to go submit an issue to JetBrains on this one.

3 Answers

Assuming this runBlocking code is called from the main thread, then the inner coroutine launched on Dispatchers.Main will never reach the front of the queue on the main thread looper because runBlocking is still blocking the main thread, so the coroutine can never start.

When you use Main.immediate, the beginning of the launched coroutine can at least run because immediate runs the coroutine up to the first suspension point if you're already on the main thread, which you are if you launched runBlocking on the main thread. When it reaches the suspend function call join(), it gets put in the main thread looper's queue and you're back to the same problem as above. Suspend function calls always break up the continuation of the coroutine.

Not answering the original question but questioning the question itself ;-).

The latch based solution "pretends" to use concurrency for both "jobs" but only the io job does. The second Runnable is just wrapping the code to run and does nothing.

This is doing the exact same thing but simpler:

val latchMain = CountDownLatch(1)
val ioExecutor = Executors.newCachedThreadPool(Executors.defaultThreadFactory())

ioExecutor.execute(
    Runnable {
        Thread.sleep(1000)
        latchMain.countDown()
    }
)

latchMain.await()

The "identical" coroutine based implementation is very simple:

runBlocking {
    val ioJob = launch(Dispatchers.IO) {
        delay(1000)
    }

    ioJob.join()
}

Or using the same thread pool for the io job:

runBlocking {
    val ioJob = launch(ioExecutor.asCoroutineDispatcher()) {
        delay(1000)
    }

    ioJob.join()
}

Note: To clear things out, one should not deliberately block the main/UI thread with runBlocking, because while the UI thread get's released inside runBlocking (if it suspends) for its child coroutines, nothing outside the runBlocking gets executed (no draw methods, nothing), which leads to frozen UI for as long as runBlocking is active.

It's probably due to how "immediate" is implemented. It's not just join(), it's any suspend function, if you call yield() it won't help, and if you call delay() mainJob will resume only when the delay is done. Basically mainJob will not resume as long as runBlocking is running, and runBlocking will not finish until mainJob is done, which is a deadlock by definition.

You can omit specifying Dispatcher.Main.immediate to the mainJob and let it inherit its context from runBlocking. And if you want to start executing mainJob as soon as its declared just yield the thread from runBlocking to it.

runBlocking {
    log("runBlocking: start")

    val ioJob = launch(Dispatchers.IO) {
        log("ioJob: About to delay")
        delay(1000)
        log("ioJob: Delay done")
    }

    val mainJob = launch {
        log("mainJob: About to join ioJob")
        ioJob.join()
        log("mainJob: ioJob successfully joined")
    }


    // yield() if you want to start executing mainJob before the rest of the runBlocking code

    log("runBlocking: about to join jobs")
    listOf(ioJob, mainJob).joinAll()
    log("runBlocking: jobs joined, exiting")
}

private fun log(msg: String) = Log.i("MainActivity", "[${Thread.currentThread().name}] $msg")

w/o yield()

I/MainActivity: [main] runBlocking: start
I/MainActivity: [main] runBlocking: about to join jobs
I/MainActivity: [DefaultDispatcher-worker-1] ioJob: About to delay
I/MainActivity: [main] mainJob: About to join ioJob
I/MainActivity: [DefaultDispatcher-worker-3] ioJob: Delay done
I/MainActivity: [main] mainJob: ioJob successfully joined
I/MainActivity: [main] runBlocking: jobs joined, exiting

enter image description here

Related