I have a large index that contains data in the following format
{
"metadata": {"timestamp": "1970-01-01T00:00:00Z"},
"input": "input_1",
"output": ["a", "b", "c"]
}
Sometimes the same input results in different output e.g
{
"metadata": {"timestamp": "1970-01-01T00:00:00Z"},
"input": "input_1",
"output": ["a", "b"] <---- Missing "c"
}
{
"metadata": {"timestamp": "1970-01-01T00:00:00Z"},
"input": "input_1",
"output": ["a", "b", "c", "d"] <----- "d" Added
}
I want to visualize only the upward trend grouped by the input meaning if
input_1 resulted in a, b, c at T then, input_1 resulted in a, b in T+1.
I want to visualize at T and T+1 the number of outputs related to input_1 is 3.
keep in mind I'm looking for a unique count.
Please note that I'm looking for a way without reindexing or using pipelines as the data is already there and enormous.
I'm not an Elasticsearch expert so I hope I'm using the correct terms here, and the solution to my problem can be done within elasticsearch. I tried looking online but most of the guides talk about a cumulative sum and I'm looking for a unique cumulative sum