Move mutable reference out of mutable object

I have an iterator over an enum that has a mutable reference in one of its variants. Now I want to move this reference out self and return it. To avoid having two mutable references to the same object at the same time, I want to change the enum variant of self to one that does not have the reference. Below is a code example:

enum Test<'a> {
    A(&'a mut usize),
    B,
}

impl<'a> Iterator for Test<'a> {
    type Item = &'a mut usize;

    fn next(&mut self) -> Option<Self::Item> {
        match *self {
            Test::A(r) => Some(r), // TODO: return mutable reference and change self to B
            Test::B => None,
        }
    }
}

fn main() {
    let mut v = 1;
    let mut it = Test::A(&mut v);
    it.next();
}

The question is related to Change selector in match when selector is a mutable reference, but that solution does not work here.