Why does this scheduler block Requests and how can it be prevented?

Viewed 26

Our Kotlin(1.7.10)-based reactive Springboot(2.7.3) application uses a Scheduler like this to schedule Notifications for the frontend:

@Component
class NotificationScheduler(
  val transactionManager: ReactiveTransactionManager,
  val time: Time,
) {
  companion object {
    const val DELAY_BETWEEN_SCHEDULES = 5000L // milliseconds
  }

  val logger = logger()

  @Scheduled(
    fixedDelay = DELAY_BETWEEN_SCHEDULES
  )
  fun scheduler() {
    Mono.fromRunnable<Unit> {
      /* the transaction is required to keep row locks for the duration of the transaction
            to sync this scheduler to those of other instances */
      runBlocking {
        val tx = TransactionalOperator.create(transactionManager)
        tx.executeAndAwait {
          try {
            ...
          } catch (e: Exception) {
            logger.error("NotificationCron rollback", e)
            it.isRollbackOnly
          }
          return@executeAndAwait
        }
      }
    }
      // this ensures that the above code runs on a different
      // thread from the selected thread pool
      .subscribeOn(Schedulers.boundedElastic())
      .subscribe()
  }
}

The actual code inside the try-catch-block is quite computation intensive and includes a lot of Database-operations, both writing and reading. The intent is that each backend instance does this work on the side while still processing Requests from the frontend. However, we found that the backend becomes unresponsive while it is doing this scheduling work.

We haven't managed to determine the cause of this. Is there a problem with the above code or is there a bottleneck elsewhere?

0 Answers
Related