NestedScrollView no longer working as soon as I add SwipeRefreshLayout

I noticed a strange behaviour in my application. My scrollview contains two elements.

1. a SearchView
2. a RecyclerView

Those are wrapped by an LinearLayout. Now I want to add a SwipeRefreshLayout to refresh the data containing in the RecyclerView. As soon as I do that the ScrollView ignores the SearchView and only scrolls through the RecyclerView.

Here is the XML:

<androidx.core.widget.NestedScrollView
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:fillViewport="true">



        <LinearLayout
            android:layout_width="match_parent"
            android:layout_height="wrap_content"
            android:orientation="vertical">


            <SearchView
                android:id="@+id/search_bar"
                android:layout_width="match_parent"
                android:layout_height="wrap_content"
                android:iconifiedByDefault="false"
                android:layout_marginTop="45dp"
                android:queryHint="Search Here" />

            <androidx.swiperefreshlayout.widget.SwipeRefreshLayout
                android:id="@+id/refresh"
                android:layout_width="match_parent"
                android:layout_height="match_parent">

            <androidx.recyclerview.widget.RecyclerView
                android:id="@+id/listRV"
                android:layout_width="match_parent"
                android:layout_height="match_parent"
                android:visibility="gone" />

            </androidx.swiperefreshlayout.widget.SwipeRefreshLayout>

        </LinearLayout>

</androidx.core.widget.NestedScrollView>

It should scroll the SearchView and the RecyclerView at the same time. Why does it stop working as soon as I add a SwipeRefreshLayout and how can I fix that?