How can i perform a complex computation within a elasticsearch ingest pipeline script?

estransconder 0
Viewed 19

I'm attempting to index very large numbers (WEI) into Elasticsearch, however due to the limitations of mapping types (uint64 & int32) it would be problematic, considering that we want to run sum aggregations on these fields.

So the only thing that makes sense would be to convert these values to whole numbers instead of WEI. There are two parts to this question/issue, the first is that we are indexing the amount in WEI, we know ETH's decimals are 18. So it's a simple equation of:

value / 10 ^ decimals

In this decimals would be 18. The output of the above equation on let's say 1000000000000000000000 is 1000

Is it possible to perform this through ingest pipelines? If so how?

The second part of this, is certain fields have their own decimals (non-eth payments), and that decimal is in the document (in another field, same index) is it possible to perform this multiplication based on that value? On a per-document basis as well for those specific fields through ingest pipelines?

I'm on Elasticsearch 7.10

elasticsearch
1 Answers

tldr;

Can I do complex computation in ingest pipeline ?

Yes, using the ingest processor leveraging painless you should be able to write down the computation you want.

Will elasticsearch be able to handle such a degree of precision ?

Elasticsearch is able to work with those numeric types So as long as it fits in one of those, you should have no issue.

Can I reference value from the same document but other fields ?

Yes, absolutely. The processor comes with some pre define variable (a context) holding some data usually from the documents you are working with.

userxL1j0
Related