How to duplicate Rust Stream

Viewed 971

I have a program similar to the following:

struct MyEvent { /* some fields */ }
struct MyStruct { /* some fields */ }
struct MyStreamer { /* holds some state */ }

impl MyStreamer {
    pub fn stream_objects<'a, 'b: 'a>(
        &'a self,
        event_stream: Pin<Box<dyn Stream<Item = MyEvent> + Send + 'b>>,
    ) -> Pin<Box<dyn Stream<Item = Arc<MyStruct>> + Send + 'a>> { /* implementation */ }
}

The goal is to process events and build a stream of MyStruct. I then have two consumers for the stream of MyStruct and I'm struggling to duplicate it.

I'm trying to write the following function (also see my attempted implementation):

pub fn duplicate_stream<'a, 'b: 'a>(
    &'a self,
    struct_stream: Pin<Box<dyn Stream<Item = Arc<MyStruct>> + Send + 'b>>,
) -> (
   Pin<Box<dyn Stream<Item = Arc<MyStruct>> + Send + 'b>>,
   Pin<Box<dyn Stream<Item = Arc<MyStruct>> + Send + 'b>>
) {
        let (s1, r1) = mpsc::unbounded::<Arc<MyStruct>>();
        let (s2, r2) = mpsc::unbounded::<Arc<MyStruct>>();

        let s = s1.fanout(s2);

        let _handle = tokio::spawn(async move { struct_stream.map(Ok).forward(s).await });

        (r1.boxed(), r2.boxed())
}

At this point, I'm told the following:

    |
155 |         struct_stream: Pin<Box<dyn Stream<Item = Arc<MyStruct>> + Send + 'b>>,
    |                               ----------------------------------------------- this data with lifetime `'b`...
...
165 |         let _handle = tokio::spawn(async move { struct_stream.map(Ok).forward(s).await });
    |                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...is captured here...
    |
note: ...and is required to live as long as `'static` here
   --> *file name here*
    |
165 |         let _handle = tokio::spawn(async move { struct_stream.map(Ok).forward(s).await });
    |                       ^^^^^^^^^^^^

I can remove the lifetimes, but then static will be inferred and I get errors in the callers.

I'm curious to understand what the best way is to clone all the elements of a stream and obtain two identical streams. Using tokio::spawn and mpsc channels seems to require changing a lot of lifetimes to static.

1 Answers

I think this is the XY problem

Compiler is correct and you probably should think about the lifetimes because the fanout is ok.

tokio::spawn requires 'static and you have 'b lifetime specified for your struct_stream. Maybe wrap the struct_stream in an Arc/Rc?

Related