Rails & postgres collection of random records with pagination and custom weight on specific column

user_11058865 0
Viewed 150

I would like to retrieve a random collection of items paginated with a particular weight on the created_at.

I successfully retrieved a random collection paginated with postgres option setseed.

The thing is, how do I combine some sort of weighing on created_at in my collection (which will give a better chance for the weighed items to be in the random sample) and this setseed option with postgres.

I'm thinking of something like retrieving the items, add them the weight I want and then do my random request but I think it will not be good performance-wise.

I'm in a kind of a dead end there and I don't know how to approach this issue.

Here is what I did for now : Simply using setseed option to have a different batch of random items on each of my pages :

Item.connection.execute "select setseed(0.5)"
Item.where(...).order('random()').page(params[:page]).per_page(15)
random-seed postgresql ruby-on-rails
2 Answers

I would suggest to convert your created_at to a a float. Here is an example

Item.select("*, RANDOM() * to_char(created_at, 'YYYYMMDD')::float AS my_new_order_val").order(my_new_order_val: :desc)
user_26164740

Randomized sorting with weight on the created_at timestamp can be achieved with some math.

The random() function in postgres will always create a value where 0.0 <= random() < 1.0.

Since you want the newest items first, create a newness ratio so that anything created just now has a ratio of 1/1 or 100%.

Anything older than just now has a lower newness ratio than 100%.

For example, if now() in epoch time is 1645955465 and yesterday is 1645869065, and a year ago is 1614419721, the ratios are:

now/now is  1645955465/1645955465 = 1.0
yesterday/now is 1645869065/1645955465 = 0.99994
1 year ago/now is 1614419721/1645955465 = 0.98084

The ratio calculation above may work for you. In the calculation above, now is 100% new, yesterday is 99.994% new, and one year ago is 98.084% new.

Next, multiply the newness ratio by a random number. This gives you a weighted random number. Newer items will have more weight. To do the calculation, extract the newness ratio epoch and multiply by a random number.

Item.where(...)
  .order
  ("(extract(epoch(from created_at)) 
    / extract(epoch from now())) 
    * RANDOM()")
  .page(params[:page])
  .per_page(15)

Depending on your data, the ratios might not be different enough to have a noticeable effect on the random number sort. There are many ways to manipulate the randomized sort beyond what's described above. For example, you could reduce the amount of randomness by giving the randomizer a smaller range than 0 to 1. Or you could make the newness ratios have a greater range than 0.98084 to 1.

user_18635920
Related