PagerAdapter and ViewPager start a new fragment over the current visible view

I have a ViewPager with a PagerAdapter that is called from another fragment, a main fragment if you will.
My PagerAdapter is showing the user images and the user can swipe between them using ViewPager. I've registered an onClick listener for every image and it seems to work as expected, at least it returns the correct position if I debug the position from instantiateItem. The problem is that this onClick is supposed to start a new fragment, like an overlay over the image that the user clicked on but it doesn't start the fragment over the current image that the user clicked on for some reason.

Example of what I mean:

On page one onClick it starts the fragment at the correct position, (overlaying the current visible view).
On page two onClick it starts the fragment on page 1 (not in the visible view and not where the onClick fired).
On page three onClick it starts the fragment on page 2 (not in the visible view and not where the onClick fired).

Code:

MainFragment.java

public class MainFragment extends Fragment implements AdapterView.OnItemSelectedListener {
ViewPager viewPager;

    @Override
    public void onViewCreated(View view, @Nullable Bundle savedInstanceState) {
    viewPager = (ViewPager) getView().findViewById(R.id.viewPager);
    swipeAdapter = new SwipeAdapter(getActivity(), getContext(), stringFetchedName, stringFetchedUserID, stringFetchedUsername, stringFetchedRating, stringFetchedDescription, stringFetchedDate);
    viewPager.setAdapter(swipeAdapter);
    }
}

SwipeAdapter.java

public class SwipeAdapter extends PagerAdapter {
    FragmentActivity activity;
    Context context;
    String names[], userid[], username[], rating[], description[], date[];
    LayoutInflater layoutInflater;
    ImageView imageView;
    public View itemView;

  public SwipeAdapter(FragmentActivity activity, Context context, String names[], String userid[], String username[], String rating[], String description[], String date[]) {
        this.activity = activity;
        this.context = context;
        this.names = names;
        this.userid = userid;
        this.username = username;
        this.rating = rating;
        this.description = description;
        this.date = date;
        layoutInflater = (LayoutInflater) context.getSystemService(Context.LAYOUT_INFLATER_SERVICE);
    }

   @Override
    public int getCount() {
        return names.length;
    }

   @Override
    public boolean isViewFromObject(View view, Object object) {
        return view == ((View) object);
    }

   @Override
    public Object instantiateItem(final ViewGroup container, final int position) {
        final View itemView = layoutInflater.inflate(R.layout.imageswap, container, false);
        imageView = itemView.findViewById(R.id.imageView);
        container.addView(itemView);

        imageView.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                if (itemView.findViewById(R.id.userText).isShown()) {
                    showImageOverlay(v);
                } else {
                    hideImageOverlay(v);
                }
            }
        @Override
        public void destroyItem(ViewGroup container, int position, Object object)    {
            container.removeView((View) object);
           }
       }

      public void showImageOverlay(View view) {
        Fragment fragment = new ImageOverlayFragment().newInstance("0", "0");
        FragmentManager fragmentManager = activity.getSupportFragmentManager();
        FragmentTransaction fragmentTransaction = fragmentManager.beginTransaction();
        fragmentTransaction.replace(R.id.imageSwap, fragment, IMAGE_OVERLAY);
        fragmentTransaction.addToBackStack(null);
        fragmentTransaction.commit();
    }

      public void hideImageOverlay(View view) {
        Fragment fragment = activity.getSupportFragmentManager().findFragmentByTag(IMAGE_OVERLAY);
        if(fragment != null) {
 activity.getSupportFragmentManager().beginTransaction().remove(fragment).commit();
        }

    }
}

How can I make sure that the new fragment ImageOverlayFragment I start is started on top of the image that was clicked, like how could I take the position from instantiateItem into account etc? or what should I do?

If I debug Log.d("debug", "Clicked position:" + position); inside the imageView OnClickListener it gives me the correct position but when I call showImageOverlay(v), it seems like it always starts the fragment on the view to the left of my currently visual view?

Note: I'm passing a FragmentActivity to the SwipeAdapter from the MainFragment which is why I can access getSupportFragmentManager().

Update:

I tried to use FragmentStatePagerAdapter instead of PagerAdapter as suggested by SO and the result is still the same. The onClick event is sending the correct position and I can toggle the visibility of visible elements within the onClick event etc but I still cannot start the new fragment over the view of where the user clicked. The new fragment still starts to the left, and not in the view of where the user clicked. Problem still remains.