popBackStack() is not working in android NavController

I am using android JetPack navigation to navigate between my fragments. This is how my navigation graph looks like

 <?xml version="1.0" encoding="utf-8"?><navigation xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:app="http://schemas.android.com/apk/res-auto"
xmlns:tools="http://schemas.android.com/tools"
android:id="@+id/registration_nav_graph"
app:startDestination="@id/registrationStoreNameFragment">

<fragment
    android:id="@+id/registrationStoreNameFragment"
    android:name="com.minidukaan.android.feature.registration.view.RegistrationStoreNameFragment"
    android:label="fragment_registration_store_name"
    tools:layout="@layout/fragment_registration_store_name" >
    <action
        android:id="@+id/action_registrationStoreNameFragment_to_registrationStoreUrlFragment2"
        app:destination="@id/registrationStoreUrlFragment2" />
</fragment>

<fragment
    android:id="@+id/registrationStoreUrlFragment2"
    android:name="com.minidukaan.android.feature.registration.view.RegistrationStoreUrlFragment"
    android:label="RegistrationStoreUrlFragment" >
    <action
        android:id="@+id/action_registrationStoreUrlFragment2_to_registrationLocationFragment"
        app:destination="@id/registrationLocationFragment" />
    <argument
        android:name="store_name"
        app:argType="string" />
    <argument
        android:name="store_slug"
        app:argType="string" />
</fragment>


<fragment
    android:id="@+id/registrationLocationFragment"
    android:name="com.minidukaan.android.feature.registration.view.RegistrationLocationFragment"
    android:label="fragment_registration_location"
    tools:layout="@layout/fragment_registration_location" >
    <action
        android:id="@+id/action_registrationLocationFragment_to_registrationSuccessFragment"
        app:destination="@id/registrationSuccessFragment" />
    <argument
        android:name="store_name"
        app:argType="string" />
    <argument
        android:name="store_slug"
        app:argType="string"
        android:defaultValue=""/>
</fragment>
<fragment
    android:id="@+id/registrationSuccessFragment"
    android:name="com.minidukaan.android.feature.registration.view.RegistrationSuccessFragment"
    android:label="fragment_registration_success"
    tools:layout="@layout/fragment_registration_success" />

From RegistrationStoreNameFragment I navigate to RegistrationStoreUrlFragment like this

private lateinit var navController: NavController
override fun init(savedInstanceState: Bundle?) {
    super.init(savedInstanceState)
    ....

    binding.lifecycleOwner = this
    navController = view?.let { Navigation.findNavController(it) }!!
    val action = RegistrationStoreNameFragmentDirections.
                        actionRegistrationStoreNameFragmentToRegistrationStoreUrlFragment2(storeName, it.slug)
                            navController.navigate(action)

}

so it navigates perfectly, but when I do navController.popBackStack() in RegistrationStoreUrlFragment then it doesn't navigate back. Any help will be highly appreciated as I am stuck in this issue for a long time.

For other fragments popBackStack is working fine but not for the ones mentioned above.