Rust: How to filter out "None" emissions?

user_11392896 0
Viewed 3260

My use-case is that I want to scan over an iterator, and yield accumulated values over segments of the original iterator (this is for a tokenizer). In other words, it's not a 1-to-1 mapping between the input values and output values. Note that filter_map() won't work because I do need the accumulator value.

I found .scan(), which is almost what I want:

#![allow(unused)]
fn main() {
    let a = [1, 2, 3];
    
    let mut iter = a.iter().scan(1, |state, &x| {
        if x == 2 {
            return None;
        }
    
        // each iteration, we'll multiply the state by the element
        *state = *state * x;
    
        // then, we'll yield the negation of the state
        Some(-*state)
    });
    
    println!("{:?}", &iter.next());
    println!("{:?}", &iter.next());
    println!("{:?}", &iter.next());
}

Except that the above outputs

Some(-1)
None
Some(-3)

When I want it to output

Some(-1)
Some(-3)
None

And, despite what you might think, this doesn't work:

        Some(-*state)
    }).filter(|x| x.is_some());

Because I'm not actually iterating over Options:

error[E0599]: no method named `is_some` found for reference `&{integer}` in the current scope
  --> src/main.rs:15:21
   |
15 |     }).filter(|x| x.is_some());
   |                     ^^^^^^^ method not found in `&{integer}`

So it's like iterator methods are intentionally shielded from the "missing yield value" case.

Any ideas how I can a) filter out those missing yields, or b) accomplish the above in some totally different way?

rust iterator
2 Answers

You can use filter_map and make your own accumulator in a variable outside the iterator:

fn main() {
    let a = [1, 2, 3];

    let mut state = 1;

    let mut iter = a.iter().filter_map(|&x| {
        if x == 2 {
            return None;
        }

        // each iteration, we'll multiply the state by the element
        state = state * x;

        // then, we'll yield the negation of the state
        Some(-state)
    });

    println!("{:?}", &iter.next()); // Some(-1)
    println!("{:?}", &iter.next()); // Some(-3)
    println!("{:?}", &iter.next()); // None
}
user_33988390

Just printing 3 iter.next() should be a very uncommon use case. Mostly one would loop until None or use a function like fuse or collect on the iterator. Each of these functions cuts all results after the first None. So returning None from an iterator means: terminate the iterator.

Consequently you should avoid to terminate the iterator in your scan closure, e.g.

    let mut iter = a.iter().scan(1, |state, &x| {
        if x == 2 {
            return Some(None); // continue, but without value
        }
    
        // each iteration, we'll multiply the state by the element
        *state = *state * x;
    
        // then, we'll yield the negation of the state
        Some(Some(-*state)) // continue with value
    }).flatten(); // remove all missing values

or by separating the scan from the filter:

let mut iter = a
    .iter()
    .scan(1, |state, &x| {
      if x == 2 {
        return Some((x, *state));
      }

      // each iteration, we'll multiply the state by the element
      *state = *state * x;

      // then, we'll yield the negation of the state
      Some((x, -*state))
    })
    .filter_map(|(x, state)| if x == 2 { None } else { Some(state) });
user_44972530
Related