Keyset Pagination with Null Values

Viewed 656

I have been trying to get Keyset Pagination to work with a query that has multiple nullable columns in the sort.

Everything sorts as expected, The problem is when you start doing the keyset pagination over the values. The problem is clearly with the fact that you can not do normal comparisons against null

select null < 'A' returns null (not true or false)

so when you do (last_name, first_name, middle_name, id) > (null, 'Elinor', null, null) in your where clause it evaluates to null

This creates issues when I am trying to display people that have no names or no last name, I cannot paginate through those pages.

I tried coalesce a 'ZZZZZZZZZ' to the fields, but that seemed to create weird artifacts in some places that was causing my pagination to reset and start over

I do not care about the order of where the nulls appear (first or last) as long as it is consistent and I can get a good consistent working pagination.

In the real system I am left joining the names on another table so the name fields could all be null (The id in that case in the other table's id, so it always has a value)

create table names(
    id serial primary key,
    people_id integer,
    last_name text,
    first_name text,
    middle_name text
)

-- A person with no last name needs to be able to be paged
select * from names
where (last_name, first_name, middle_name, id) > (null, 'Elinor', null, null)
order by last_name, first_name, middle_name, id
limit 1

-- A person with no name needs to be able to be paged 
select * from people
left join names on people.id = names.people_id
where (last_name, first_name, middle_name, people.id) > (null, null, null, 200)
order by last_name, first_name, middle_name, id
limit 1

1 Answers

The idea with coalesce is the correct one.

It looks like you are missing a unique not nullable column (id) at the end of the key set. That is required to keep the columns unambiguous so that you don't experience those weird "start over" effects.

Make sure to include the coalesce expressions in the index you are using to speed up searching and sorting.

Related