Inconsistent results in filtered queries in Weaviate. HNSW graph traversal + filtering

Viewed 47

I have setup a Weaviate db witch holds about 12M vectors, along with some metadata for each one.

I am getting inconsistent/wrong/weird??? results when I perform filtered search, i.e. filter on a meta data field and then perform ANN search. (I have turned brute force search off completely by passing in a very small number to the "flatSearchCutoff" parameter '500').

Each vector has a 'user' field attached to it which is set to a 'string' type, moreover there is a 'status_int' field which takes the values of [0,4]. There is also a unique identifier which is another 'string' field.

Firstly my use case requires to query the DB for a specific entry and retrieve the vector representation...by using the unique identifier field, which I do with the following:

where_filter = {
  "path": ["resource_identifier"],
  "operator": "Equal",
  "valueString": identifier_var
}
result = client.query.get("Resource", ["resource_identifier" ])\
                    .with_where(where_filter)\
                    .with_additional(['vector'])\
                    .with_limit(1)\
                    .do()
feature_vector = np.array(result['data']['Get']['Resource'][0]['_additional']['vector'],dtype=np.float32)
nearVector = {"vector":feature_vector, "certainty":SIMILARITY_THRESHOLD}

This works really well - and is blazing fast.

Secondly I want to search for nearby vectors to the one I just retrieved... while making sure that the results fit my criteria. I am applying the following filter using the metadata fields.

filter_ ={
        'operator':'And',
        'operands':
        [
      
            {
                'path': 'user',
                'operator': 'Equal',
                'valueString': user_var
            },
            {
                'path': 'status_int',
                'operator': 'GreaterThanEqual',
                'valueInt': status_var
            }
        ]
        }


result = client.query.get("Resource", ["resource_uri","user", "_additional{certainty}",'status_int'])\
            .with_where(filter_)\
            .with_near_vector(nearVector)\
            .with_limit(RETRIEVE_RECORDS_LIMIT).do()

While the filtering process does not throw any errors...the results look... weird...I set the similarity threshold to 0.75, so that I get plenty of results... However the results only include really close matches to the query vector... even at a very small similarity threshold which I found really odd.

More specifically I query the DB as follows:

  1. When I query for a specific user and status_int >= 0 i'm stuck with 2 identical results of similarity 1. However there should be a lot more results.... since the filter covers 3295 objects.

  2. When I query for the same user and status_int >= 1 I get 1 resource as a result... again with similarity=1, which is one of the 2 results i'm given above...( this filter encompasses 2578 objects)

  3. HOWEVER when I query for the same user and status_int >= 2 I GET ALOT MORE RESULTS ! With no exact matches (as it should be) but with 0.85 and below similarity. (this filter encompasses 1900 resources)

So my question is.... isn't this weird or is this intended behaviour and I've misunderstood how Weaviate and HNSW work?

-In my mind the path through the NHSW graph is the same across these queries... its just that results are retrieved based on the filters passed...

Shouldn't the first 2 queries present the perfect matches AS WELL AS the less relevant matches of the third query?

Really confused on this one :(

0 Answers
Related