Multiple Panes with MouseListener on top of each other

I have an issue with mouse listeners added to 2 different panels one on top of the other. Both of them make use of the mouseEntered() and mouseExited() methods.

The expected outcome of the test code bellow is:

• When I hover on the frame a red rectangle should be made visible
• When I hover on the red rectangle in the frame it should turn blue.
• When my mouse exits the now blue rectangle (but still inside the frame) it should turn red
• When my mouse exits the frame entirely the now red rectangle should not be visible

If I try to move my mouse over the colored rectangle it keeps flashing from visible to not visible. The system print shows what I mean, it keeps activating Mouse Entered and Mouse Exited every time I move or click with my mouse. Is there anyway to prevent the mouseExited() method from being called every time I move or click my mouse?

Here is the test code:

import java.awt.BorderLayout;
import java.awt.Color;
import java.awt.Dimension;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import javax.swing.JFrame;
import javax.swing.JPanel;

public class Sandbox extends JPanel {

    public static void main(String[] args) {
        JFrame frame = new JFrame();
        frame.setSize(500, 500);
        frame.add(new Sandbox());
        frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
        frame.setVisible(true);

    }

    public Sandbox() {
        super(new BorderLayout());

        final JPanel panelA = new JPanel();
        panelA.setBackground(Color.red);
        panelA.setPreferredSize(new Dimension(155, 155));
        panelA.setVisible(false);

        this.add(panelA, BorderLayout.WEST);

        this.addMouseListener(new MouseAdapter() {
            @Override
            public void mouseEntered(MouseEvent e) {

                System.out.println(" -  MOUSE ENTERED ");
                panelA.setVisible(true);
            }

            @Override
            public void mouseExited(MouseEvent e) {

                System.out.println(" -  MOUSE EXITED ");
                panelA.setVisible(false);
            }

        });

        panelA.addMouseListener(new MouseAdapter() {

            @Override
            public void mouseEntered(MouseEvent e) {

                System.out.println(" #  MOUSE ENTERED ");
                panelA.setBackground(Color.blue);

            }

            @Override
            public void mouseExited(MouseEvent e) {
                panelA.setBackground(Color.red);
                System.out.println(" #  MOUSE EXITED ");
            }

        });

    }

}