NATS KV history larger than specified when creating bucket

Question

We are using nats with KeyValue store feature (nats KV). We develop go microservices and use the nats go client. We try to leverage the history feature of nats KV with no success yet.

Certain times using nats, we retrieve a larger history than the history specified when creating the KV.

We create the KV using :

kv, _ := js.CreateKeyValue(&nats.KeyValueConfig{
        Bucket:       "some-bucket",
        Description:  "store for some-service",
        MaxValueSize: 0,
        History:      10, // should we ever get more than 10 elements when reading history ?
        TTL:          TTL,
        MaxBytes:     5000000,
        Storage:      nats.MemoryStorage,
        Replicas:     0,
        Placement:    nil,
    })

and we retrieve values using

kv.History("someId")

When we get results larger than the specified History, we get several KeyValueEntrys with the same delta value.

We are quite write intensive, and also reuse quite a lot the same key id :

we write values until a certain point,
call kv.Purge("someId")
and then we may reuse "someId" later on in the process.

Writes and read are asynchronous and concurrent.

Here is our client go.mod regarding nats:

    github.com/nats-io/nats-server/v2 v2.8.4
    github.com/nats-io/nats.go v1.16.0

and we run a nats server version 2.8.4.

note : I did not go far enough in the KV implementation details but I am worried that this is linked with jetstream. It seems like a watcher is created each time and re-reads all previous values regardless of history size. It leads me to another question : is the kv history feature appropriate for read intensive use cases ?

Thanks for your help or pointers on this matter.

NATS KV history larger than specified when creating bucket

0 Answers