I would like to calculate the similarity between two documents containing some dense vectors using a similarity score based on the L2 norm of these two vectors (their Euclidean distance). What I currently do is create my index with the following mapping:
{
"mappings": {
"properties": {
"@id": {
"type": "keyword"
},
"embedding": {
"dims": <MY_DIM>,
"type": "dense_vector"
}
}
}
}
and then execute the following query (to get, for example, 5 closest documents):
{
"size": 5,
"query": {
"script_score": {
"query": {"match_all": {}},
"script": {
"source": "1 / (1 + l2norm(params.query_vector, doc['embedding']))",
"params": {"query_vector": <MY_VECTOR>}
}
}
}
}
Instead of using script_score in my ElasticSearch queries, I would like to encode my scripted score (in my case a similarity score based on the Euclidean distance) into settings (similar to what is described here). I would like to be able to do something like this:
{
"settings": {
"similarity": {
"euclidean_similarity": {
"type": "scripted",
"script": {
"source": "1 / (1 + l2norm(params.query_vector, doc['embedding']))"
}
}
}
},
"mappings": {
"properties": {
"@id": {
"type": "keyword"
},
"embedding": {
"dims": 128,
"type": "dense_vector",
"similarity": "euclidean_similarity"
}
}
}
}
And then have some 'simple' query that does not need to specify the formula, but uses this custom euclidean_similarity provided some params.
Is there any way to do something like this in Elasticsearch without defining custom plugins? And in general, is it possible to parameterize custom scripted similarities, and if yes, how do we pass the required parameters during the query time?