I have a very simple azure search index with three fields:
- an identifier, retrievable
- keyword, string, retrievable, searchable, lucene standard analyzer
- boost, double, retrievable, filterable
It indexes a JSON array 4 documents with the exact same keyword field equal to "desc" and boost to 0.
There is no scoring profile defined so boost is transparent here.
When I search on "desc" I get 2 docs scored at 0.59 and 2 docs at 0.30.
If I add more identical documents (keyword=desc) the score evolve in the same flavor:
- 8 documents give 2 at 0.59 and 6 at 0.30
- 12 documents give 3 at 0.71, 4 at 0.59 and 5 at 0.30
I would have expected the documents scoring to have the exact same value because: - Term Frequency should be identical - Inverse Document Frequency should be identical, even if the documents are located in different shards as all documents are identical
What can explain this?
This can't be the sharding effect as all documents are similar. Is Azure Search adding a random ranking on similar match?
After further investigation adding docs one by one, I start to get what is happening. The score is dependent of the number of documents in a group (Is it what is called a shard?)
1/ the first three docs scores the same 0.30 each. Group topology is (1,1,1) 2/ adding a forth make 2 scoring at 0.59 and 2 scoring at 0.30. Group topology is (2,1,1) 3/ continuing to add we get very consistently that adding a new document moves all documents of the group it is added to the next score:
- document alone in a group scores 0.30
- documents in a group of 2 score 0.59
- documents in a group of 3 score 0.71
- documents in a group of 4 score 0.77
- documents in a group of 5 score 0.81
- documents in a group of 6 score 0.89
Then it appears that azure search does not had new groups linearly, the rate slows down:
- a group is added on the first, 2nd, 3rd, 5th, 6th, 7th, 8th, 15th, 16th, 22nd, 33rd.
- When adding lot's of same documents, scores tend to 1.
Now what could cause the Tf-Idf to depend on the number of documents in each group (Shard?) where it is applied? Could that be a constant min document in the IDF term?