Best way to use ExoPlayer for USAGE_ASSISTANT and CONTENT_TYPE_SPEECH

Viewed 110

I'm developing an application for Android in Kotlin that takes the user through a routine. They can choose to have audio assistance while they are performing the tasks. What is the best way to design this? Think something like Google Maps shooting off voice directions every once in a while.

Here are a few requirements:

  • Users can play music or podcast in the background, so each time my app speaks, the content must duck.
  • Users cannot control or interact with audio or audio queue in any way. It's really just voice assistant for the tasks they're performing.
  • Users can choose to leave the app, and use other apps, while still receiving audio cues. So the system should never kill the process.
  • The audio queue needs to be somewhat smart, so that if the user chooses to skip the step, we can remove that cue from the queue.

So far, this is the architecture that I have developed with the help of ExoPlayer documentation, UAMP sample, and RayWenderlich binding services tutorial

  • Media module in Koin injects the Audio service, download manager, and a class I use for keeping audio queue. I then call the AudioManager in my ViewModel to add items to it.
    val mediaModule = module {
        single { AudioPlayerConnection(androidApplication()) }
        single { DownloadManager(get(), get(), get()) }
        single { AudioManager(get(), get()) }
    }
  • AudioPlayerConnection is Connector class that creates AudioPlayerService, provides binding, unbinding calls, and access to service's functionality.

  • AudioPlayerService is Audio service that plays mp3 audio files downloaded from the web. It contains ExoPlayer, PlayerNotificationManager and AudioPlayerBinder. On create it basically does the following:

    override fun onCreate() {
        super.onCreate()
    
        initializeAudioPlayer() 
        audioManager = applicationContext.getSystemService(Context.AUDIO_SERVICE)
         as AudioManager
    exoPlayer?.volume = 1.0f
    val audioAttributes = android.media.AudioAttributes.Builder()
            .setUsage(android.media.AudioAttributes.USAGE_ASSISTANT)
            .setContentType(android.media.AudioAttributes.CONTENT_TYPE_SPEECH)
            .build()
    focusRequest = 
        AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK)
            .setAcceptsDelayedFocusGain(false)
            .setAudioAttributes(audioAttributes)
            .setForceDucking(true)
            .build()
    }
  • AudioManager keeps @Volatile list that is added to from the ViewModel, when the ViewModel decides to give user cues. It is responsible for downloading the url at which the audio lives, and adds the items to the list. It then pings AudioPlayerConnection that the media is ready to be played. When AudioService's players is in Player.STATE_ENDED I do the following:
    exoPlayer?.clearMediaItems()
    exoPlayer?.seekToDefaultPosition()
    if (audioManager != null && focusRequest != null) {
        audioManager!!.requestAudioFocus(focusRequest!!)
    }
    exoPlayer?.addMediaItem(item.mediaItem)

This seems rather clumsy handling of the queue in ExoPlayer, but I have found no better way.

The problems with the current solution are:

  • There is a useless notification shown for downloading the media, and playing the media. Note, the user really shouldn't know anything about audio service. They can choose in the settings to remove the cues, but other than that, no audio player functionality should be exposed to them.
  • I have a hard time keeping the reference for the items in the media queue, and removing them so they are not played if the user skips the step. How do I programmatically keep references of the objects in the exoplayer queue, that I can just remove.
  • The service does not seem to be correctly destroyed, so LeakCanary is complaining.
  • If the service crashes, I have no idea where to intercept that and create it again, so that the user can continue receiving the voice assistance.

Thank you for help and suggestions!

0 Answers
Related