Ecommerce and Vespa: filtering customer's wishlist

piranha_46854 0
Viewed 106

Let's suppose an ecommerce site has wishlisting functionality. Some users wishlist a lot of products (on a scale of tens of thousands). Total amount of products is in millions. We want to implement functionality where a customer can filter those products like in catalogue.

When we looked into implementing this using Elasticsearch, the best way we've found was using terms lookup. Like, we create a document for each user's wishlist, and then filter out products we need using this document. After that all filtering/etc is done regularly. The problem here is that Elasticsearch cannot sort those products properly — i.e. how document specifies it, since we want to sort by wishlisting time.

This is when I decided to look into Vespa. But after reading documentation I still have no idea what's the best way to implement this. This looks like a problem needing "join" to my rdbms-infected mind. :)

Cardinalities of data is like so:

  • millions of products
  • hundreds of thousands of users
  • tens of thousands of wishlisted items

So... any ideas how to implement or pointers on what to read?

vespa
2 Answers

In Vespa you need two document types for this (if you want to store the wishlist in Vespa that is, it's not a requirement)

  • Retrieved the whislist for given user, either from Vespa using get api of Vespa or from another storage solution.
  • Retrieve and rank using DotProductItem over the product id field along with a ranking profile.
/search/?yq=select * from products where dotProduct(product_id, {"a":3, "b":2});&ranking=wishlist&hits=10

In this case a is a more recent product in the wishlist then b. ranking profile to go with it:

rank-profile wishlist {
  first-phase {
    expression:rawScore(product_id)
  }
}

You can also use WAND to speed up the search and retrieve only the most recent/top ranking hits in the wishlist. The above example retrieves all and ranks all.

See

user_87142350

You can do this by using a WeightedSet item where the item tokens are an id of the products and the weight is the timestamp you want to sort by, see https://docs.vespa.ai/documentation/reference/query-language-reference.html#weightedset, or see https://docs.vespa.ai/documentation/multivalue-query-operators.html#weightedset-example to create it in a Searcher (recommended).

To rank by the timestamp, use a rank profile which just ranks by the match weight, e.g attributeMatch(name).totalWeight

(Regarding sorting, you could also just retrieve all the matches in a Searcher, resort in code, remove those below the fold and then fill() with the summary data. That will scale fine to a few tens of k as long as you do it before filling.)

user_13926040
Related