I'm developing an application for Android in Kotlin that takes the user through a routine. They can choose to have audio assistance while they are performing the tasks. What is the best way to design this? Think something like Google Maps shooting off voice directions every once in a while.
Here are a few requirements:
- Users can play music or podcast in the background, so each time my app speaks, the content must duck.
- Users cannot control or interact with audio or audio queue in any way. It's really just voice assistant for the tasks they're performing.
- Users can choose to leave the app, and use other apps, while still receiving audio cues. So the system should never kill the process.
- The audio queue needs to be somewhat smart, so that if the user chooses to skip the step, we can remove that cue from the queue.
So far, this is the architecture that I have developed with the help of ExoPlayer documentation, UAMP sample, and RayWenderlich binding services tutorial
- Media module in Koin injects the Audio service, download manager, and a class I use for keeping audio queue. I then call the
AudioManagerin my ViewModel to add items to it.
val mediaModule = module {
single { AudioPlayerConnection(androidApplication()) }
single { DownloadManager(get(), get(), get()) }
single { AudioManager(get(), get()) }
}
AudioPlayerConnectionis Connector class that createsAudioPlayerService, provides binding, unbinding calls, and access to service's functionality.AudioPlayerServiceis Audio service that plays mp3 audio files downloaded from the web. It containsExoPlayer,PlayerNotificationManagerandAudioPlayerBinder. On create it basically does the following:
override fun onCreate() {
super.onCreate()
initializeAudioPlayer()
audioManager = applicationContext.getSystemService(Context.AUDIO_SERVICE)
as AudioManager
exoPlayer?.volume = 1.0f
val audioAttributes = android.media.AudioAttributes.Builder()
.setUsage(android.media.AudioAttributes.USAGE_ASSISTANT)
.setContentType(android.media.AudioAttributes.CONTENT_TYPE_SPEECH)
.build()
focusRequest =
AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT_MAY_DUCK)
.setAcceptsDelayedFocusGain(false)
.setAudioAttributes(audioAttributes)
.setForceDucking(true)
.build()
}
AudioManagerkeeps@Volatilelist that is added to from the ViewModel, when the ViewModel decides to give user cues. It is responsible for downloading the url at which the audio lives, and adds the items to the list. It then pingsAudioPlayerConnectionthat the media is ready to be played. WhenAudioService's players is inPlayer.STATE_ENDEDI do the following:
exoPlayer?.clearMediaItems()
exoPlayer?.seekToDefaultPosition()
if (audioManager != null && focusRequest != null) {
audioManager!!.requestAudioFocus(focusRequest!!)
}
exoPlayer?.addMediaItem(item.mediaItem)
This seems rather clumsy handling of the queue in ExoPlayer, but I have found no better way.
The problems with the current solution are:
- There is a useless notification shown for downloading the media, and playing the media. Note, the user really shouldn't know anything about audio service. They can choose in the settings to remove the cues, but other than that, no audio player functionality should be exposed to them.
- I have a hard time keeping the reference for the items in the media queue, and removing them so they are not played if the user skips the step. How do I programmatically keep references of the objects in the exoplayer queue, that I can just remove.
- The service does not seem to be correctly destroyed, so LeakCanary is complaining.
- If the service crashes, I have no idea where to intercept that and create it again, so that the user can continue receiving the voice assistance.
Thank you for help and suggestions!