Navigation component not retaining its state after process death

I'm using the navigation component with kotlin DSL to build the graph and I have a weird bug, the first time that the application installs the current destination is not retained after process death, but only after installation. even if I completely close the app and clear its storage there is no problem and the current destination is retained. but the first time after installation if I'm on the second page and process death happens, it starts from the first destination. this the code for my activity:

override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)
        setContentView(R.layout.activity_main)
        configNavigation()
        ...
    }

     private fun configNavigation() {
        val navHosFragment = supportFragmentManager.findFragmentById(R.id.fragment_container) as 
            NavHostFragment
        navHosFragment.navController.apply {
            graph = createGraph(
                navGraph.id,
                navGraph.dest.places,
                GraphBuilder(applicationContext).builder
            )
        }
        val appBarConfiguration = AppBarConfiguration(navHosFragment.navController.graph)
        placesToolbar.setupWithNavController(navHosFragment.navController, appBarConfiguration)
    }

and this is the code for my second destination viewmodel

 @AssistedInject.Factory
    interface Factory {
        fun create(savedStateHandle: SavedStateHandle): DetailsViewModel
    }

    private val id = savedStateHandle.get<String>("ID") ?: ""

    init {
        getDetails(id)
    }

I tried to set graph only when savedInstanceState is null like the following code, as the documentation says that navGraph is retained across configuration changes and process death, but it gives me an exception which says "getGraph() must be called after setGraph" which means that the graph is not retained.

if (savedInstanceState == null) {
    navHosFragment.navController.apply {
            graph = createGraph(
                navGraph.id,
                navGraph.dest.places,
                GraphBuilder(applicationContext).builder
            )
        }
}