Nats Jetstream message are delivered twice

Viewed 248

I'm new to Nats thank you for your help. I want one stream(main bus) that gets all the signals(signals.>), and one consumer that gets all the signals from the East side(signals.east.>). A couple of applications to consume from that consumer. Each application may have 1-5 instances, but I only want one of those instances to receive that signal(no duplication and order matters).

I have a stream with the following config:

Configuration:

             Subjects: signals.>
     Acknowledgements: true
            Retention: File - Interest
             Replicas: 1
       Discard Policy: Old
     Duplicate Window: 40s
    Allows Msg Delete: true
         Allows Purge: true
       Allows Rollups: false
     Maximum Messages: unlimited
        Maximum Bytes: unlimited
          Maximum Age: 40.00s
 Maximum Message Size: unlimited
    Maximum Consumers: unlimited

Cluster Information:
                 Name: test-east
               Leader: nats-0

State:
             Messages: 0
                Bytes: 0 B
             FirstSeq: 632
              LastSeq: 631 @ 2022-07-26T23:58:13 UTC
     Active Consumers: 10

Here is config for consumer:

Configuration:

        Durable Name: e5
    Delivery Subject: _INBOX.SZqS5641roDOlg7tlbea4w
      Filter Subject: signals.east.test.>
      Deliver Policy: All
 Deliver Queue Group: e5
          Ack Policy: Explicit
            Ack Wait: 30s
       Replay Policy: Instant
     Max Ack Pending: 1
        Flow Control: false

Cluster Information:

                Name: test-east
              Leader: nats-0

State:

   Last Delivered Message: Consumer sequence: 13 Stream sequence: 591 Last delivery: 33m17s ago
     Acknowledgment floor: Consumer sequence: 13 Stream sequence: 591 Last Ack: 33m17s ago
         Outstanding Acks: 0 out of maximum 1
     Redelivered Messages: 0
     Unprocessed Messages: 0
          Active Interest: No interest

I tried this code:


    sub, err := js.PullSubscribe(subj, consName)
    if err != nil {
        fmt.Println(err)
        return
    }

    if err != nil {
        log.Fatalf("Error setting pending limits on the subscriber: %v", err)
    }

    ctx := context.TODO()

    for {
        select {
        case <-ctx.Done():
            return
        default:
        }
        msgs, err := sub.Fetch(1, nats.Context(ctx))
        for _, msg := range msgs {
            msg.AckSync()
            if err != nil {
                fmt.Println(err)
                log.Fatal(err)
            }
            fmt.Printf(fmt.Sprintf("pull-sub Msg:%s- %s\n", msg.Header.Get(nats.MsgIdHdr), string(msg.Data)))
        }
    }

but each instance gets same message twice no matter what. also i tried this code:

func jetsubscribeConsumer(js nats.JetStreamContext, subj, queue string) (err error) {
    ctx := context.TODO()

    handler := func(m *nats.Msg) {
        m.AckSync()
        mdata, _ := m.Metadata()
        fmt.Println(mdata.Stream, mdata.Consumer, mdata.Domain, mdata.NumDelivered, mdata.NumPending, mdata.Sequence.Consumer)
        fmt.Println(queue, string(m.Data))
    }

    _, err = js.QueueSubscribe(subj, queue, handler, nats.MaxAckPending(1), nats.ManualAck())
    if err != nil {
        fmt.Println(err)
        return
    }

    <-ctx.Done()

    return nil
}

here is the result for instance one of application a

pull-sub Msg:0- xyz 0- 16:57:05
pull-sub Msg:0- xyz 0- 16:57:05
pull-sub Msg:2- xyz 2- 16:57:05
pull-sub Msg:2- xyz 2- 16:57:05
pull-sub Msg:4- xyz 4- 16:57:05
pull-sub Msg:4- xyz 4- 16:57:05
pull-sub Msg:6- xyz 6- 16:57:05
pull-sub Msg:6- xyz 6- 16:57:05
pull-sub Msg:8- xyz 8- 16:57:05
pull-sub Msg:8- xyz 8- 16:57:05

here is the result for instance two of application a

pull-sub Msg:1- xyz 1- 16:58:13
pull-sub Msg:1- xyz 1- 16:58:13
pull-sub Msg:3- xyz 3- 16:58:13
pull-sub Msg:3- xyz 3- 16:58:13
pull-sub Msg:5- xyz 5- 16:58:13
pull-sub Msg:5- xyz 5- 16:58:13
pull-sub Msg:7- xyz 7- 16:58:13
pull-sub Msg:7- xyz 7- 16:58:13
pull-sub Msg:9- xyz 9- 16:58:13
pull-sub Msg:9- xyz 9- 16:58:13

and here is how i publish

func pushMsg(js nats.JetStreamContext, topic string) {
    for i := 0; i < 10; i++ {
        x := nats.NewMsg(topic)
        x.Data = []byte(
            fmt.Sprintf("xyz %v- %s", i, time.Now().Format("15:04:05")),
        )
        ack, err := js.PublishMsg(x)
        if err != nil {
            fmt.Println(err)
            return
        }
        fmt.Printf("%#v\n", ack)
        fmt.Println(i, " ", string(x.Data), x.Header.Get(nats.MsgIdHdr))
    }
}

thank you very much.

1 Answers

The configured consumer is a queue consumer that is push based and the pull based one is a separate consumer. So you have two consumers.

Related