Override zoom over swipetorefresh - Android

Okay so if I have scrolled just a little bit and the swipetorefresh gets visible, and if I try to zoom with two fingers it just gets further down to refresh. What I want to achieve is that if the user tries to zoom, the swipe to refresh to be overridden by the zoom option. Here is some code.

This is the webView simple zoom controls.

    myWebView.getSettings().setLoadWithOverviewMode(true);
    myWebView.getSettings().setUseWideViewPort(true);
    myWebView.getSettings().setBuiltInZoomControls(true);
    myWebView.getSettings().setDisplayZoomControls(false);

Here is from the SwipeToRefresh:

    mySwipeRefreshLayout.setOnRefreshListener(
         new SwipeRefreshLayout.OnRefreshListener() {
              @Override
              public void onRefresh() {
                   myWebView.reload();
                   }
         });

And I have mySwipeRefreshLayout.setRefreshing(true); and mySwipeRefreshLayout.setRefreshing(false); depending where I want to use them. Any suggestions?