Rxjava polling with observable

I would need to do an API call every 1 second and keep retrying while the "status" is "in_progress". However, every time I do the API call, I would receive a new QR code which I'd need to show in the UI. The call stops when "status" becomes "successful"

The response data class is

data class PollingData(
    val status: Status,
    val qrCode: String?,
)

I have this code in my repository which takes care of the 1-sec call

override fun poll(id: String): Observable<PollingData> {
    return api.poll(id)
        .subscribeOn(schedulerProvider.io())
        .flatMap {
            return@flatMap when (response.status) {
                "successful" -> Observable.just(it)
                "failed" -> Observable.error(VerificationFailedException())
                "in_progres" -> Observable.error(NotReadyException())
            }
        }
        .retryWhen {
            it.flatMap { error ->
                when (error) {
                    is NotReadyException -> Observable.just(Any()).delay(1, TimeUnit.SECONDS)
                    else -> Observable.error(error)
                }
            }
        }
}

private class NotReadyException : Exception()
private class VerificationFailedException : Exception()

In my viewmodel I call the poll function

private fun pollForResponse(id: String) {
    repository.poll(id)
        .subscribeOn(schedulerProvider.io())
        .observeOn(schedulerProvider.ui())
        .subscribe(
            {
                when (response.status) {
                    "in_progress" -> response.qrCode?.let { _qrCodeLiveData.value = it } // update qr code ui
                    "successful" -> // done
                    "failed" -> // failed
                }
            },
            {
                // error
            }
        )
}

This does not work because in the repository, when the status is "in_progress" I throw NotReadyException which is handled by retryWhen. Therefore, the QR code never reaches the ViewModel to update the UI. What is the correct way of doing this?