Recycler space ItemDecoration keeps increasing during scroll

This is my first stack overflow question. I have been trying to find a solution to this problem on stack overflow in the last a few hours and I couldn't find anything.

The app (or a specific activity) I am creating here is very simple, basically a vertical recycler containing TextView and RecyclerView, something like:

Vertical RecyclerView

--- TextView (title)

--- Horizontal RecyclerView (card view)

--- TextView (title)

--- Horizontal RecyclerView (card view)

... (more goes here)

--- TextView (title)

--- Horizontal RecyclerView (card view)

(End of Vertical RecyclerView)

Everything works correctly, except when I start scrolling the vertical recycler (up and down), the first and last two horizontal RecyclerView's space item decoration (shown below) keeps increasing. I have my guess where the problem might be, but I am not sure how to fix it.

Horizontal Recycler Adapter (which is very standard)

 public class HorizontalRecyclerViewAdapter extends RecyclerView.Adapter<HorizontalRecyclerViewAdapter.ViewHolder> {
private ArrayList<String> mDataset;
private Context mContext;

// Provide a reference to the views for each data item
// Complex data items may need more than one view per item, and
// you provide access to all the views for a data item in a view holder
public class ViewHolder extends RecyclerView.ViewHolder {
    // each data item is just a string in this case
    public TextView txtViewTitle;
    public ImageView imgViewIcon;

    public ViewHolder(View itemLayoutView) {
        super(itemLayoutView);
        txtViewTitle = (TextView) itemLayoutView.findViewById(R.id.text_view);
        imgViewIcon = (ImageView) itemLayoutView.findViewById(R.id.image_view);
    }
}

// Provide a suitable constructor (depends on the kind of mDataset)
public HorizontalRecyclerViewAdapter(Context myContext, ArrayList myDataset) {
    this.mContext = myContext;
    this.mDataset = myDataset;
}

// Create new views (invoked by the layout manager)
@Override
public HorizontalRecyclerViewAdapter.ViewHolder onCreateViewHolder(ViewGroup parent,
                                                                   int viewType) {
    // create a new view
    View v = LayoutInflater.from(parent.getContext())
            .inflate(R.layout.view_image, parent, false);
    // Goes here: set the view's size, margins, paddings and layout parameters

    ViewHolder viewHolder = new ViewHolder(v);
    return viewHolder;
}

// Replace the contents of a view (invoked by the layout manager)
@Override
public void onBindViewHolder(ViewHolder holder, int position) {
    // - get element from your mDataset at this position
    // - replace the contents of the view with that element

    String mDrawableName = mDataset.get(position);
    int resID = mContext.getResources().getIdentifier(mDrawableName , "drawable", mContext.getPackageName());
    holder.imgViewIcon.setImageResource(resID);
}

// Return the size of your mDataset (invoked by the layout manager)
@Override
public int getItemCount() {
    return mDataset.size();
}

Space Item Decoration

class HorizantalSpaceItemDecoration extends RecyclerView.ItemDecoration {

private final int mHorizantalSpaceWidth;

public HorizantalSpaceItemDecoration(int horizantalItemSpace) {
    mHorizantalSpaceWidth = horizantalItemSpace;
}

@Override
public void getItemOffsets(Rect outRect, View view, RecyclerView parent,
                           RecyclerView.State state) {
    outRect.right = mHorizantalSpaceWidth;
}

Vertical Recycler Adapter:

public class VerticalRecyclerViewAdapter extends RecyclerView.Adapter<VerticalRecyclerViewAdapter.ViewHolder> {

private ArrayList<String> mHeaderList;
private ArrayList<String> mSportList;
private ArrayList<Boolean> mSpaceItemDecorationFlagList;
private Context context;

public static final int HORIZANTAL_ITEM_SPACE = 30;

// Provide a reference to the views for each data item
// Complex data items may need more than one view per item, and
// you provide access to all the views for a data item in a view holder
public class ViewHolder extends RecyclerView.ViewHolder {
    // each data item is just a string in this case
    public TextView textView;
    public CardView cardView;
    public RecyclerView recyclerView;

    public ViewHolder(View itemLayoutView) {
        super(itemLayoutView);
        cardView = (CardView) itemLayoutView.findViewById(R.id.card_view);
        textView = (TextView) itemLayoutView.findViewById(R.id.text_view);
        recyclerView = (RecyclerView) itemLayoutView.findViewById(R.id.recycler_view);
    }
}

// Provide a suitable constructor (depends on the kind of mHeaderList)
public VerticalRecyclerViewAdapter(Context context) {
    Resources res = context.getResources();
    String[] headers = res.getStringArray(R.array.header_home);
    mHeaderList = new ArrayList<String>(Arrays.asList(headers));
    String[] sports = res.getStringArray(R.array.sports);
    mSportList = new ArrayList<String>(Arrays.asList(sports));
    mSpaceItemDecorationFlagList = new ArrayList<Boolean>();
    for(int i = 0; i < mHeaderList.size(); i++){
        mSpaceItemDecorationFlagList.add(false);
    }
    this.context = context;
}

@Override
public int getItemViewType(int position) {
    // Just as an example, return 0 or 2 depending on position
    // Note that unlike in ListView adapters, types don't have to be contiguous
    return position % 2 * 2;
}

// Create new views (invoked by the layout manager)
@Override
public VerticalRecyclerViewAdapter.ViewHolder onCreateViewHolder(ViewGroup parent,
                                                                 int viewType) {
    // Create a new view depends on view type
    // 0: Text View (Header)
    // 1: Recycler View (Category Explorer)
    View newView;

    if(viewType == 0) {
        newView = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.view_text, parent, false);
    }else{
        newView = LayoutInflater.from(parent.getContext())
                .inflate(R.layout.view_recycler, parent, false);
    }
    // Goes here: set the view's size, margins, paddings and layout parameters

    ViewHolder viewHolder = new ViewHolder(newView);
    return viewHolder;
}

// Replace the contents of a view (invoked by the layout manager)
@Override
public void onBindViewHolder(ViewHolder holder, int position) {

    // - get element from your mHeaderList at this position
    // - replace the contents of the view with that element
    if(position % 2 == 0)
    {
        String header = mHeaderList.get(position);
        holder.textView.setText(header);
    }else{
        // ??????????????
        // Problem might be holder.recyclerView not reset properly?

        RecyclerView.LayoutManager popularLayoutManager= new LinearLayoutManager(context, LinearLayoutManager.HORIZONTAL, false);
        RecyclerView.Adapter adapter = new HorizontalRecyclerViewAdapter(context, mSportList);
        holder.recyclerView.setLayoutManager(popularLayoutManager);
        holder.recyclerView.setAdapter(adapter);

        //add Space ItemDecoration
        holder.recyclerView.addItemDecoration(new HorizantalSpaceItemDecoration(HORIZANTAL_ITEM_SPACE));
        holder.recyclerView.setHasFixedSize(true);

    }
}

// Return the size of your mHeaderList (invoked by the layout manager)
@Override
public int getItemCount() {
    return mHeaderList.size();
}

In each horizontal recycler, I am trying to add 30 dp space between each image/item, but during scrolling, the first horizontal and last two horizontal recycler views kept increasing. (30, then 60 then 90 I am guessing?)

my guess is in onBindViewHolder(), my holder's recycler view is not reset properly, and its rect.right keeps getting updated to a new value (old + 30). But this is purely my guessing. Could anyone else please?

Thanks in advance.