Pagination

From BrAPI-Wiki
Jump to navigation Jump to search

General

The pagination object is applicable only when the result payload contains a data field. It describes the pagination of the records contained in the data array, as a way to identify which subset of data is being returned.

If the data key is not present in the results, pagination should be ignored. All of the following responses are acceptable, and should be ignored equally.

pagination key omitted from the metadata
"pagination": null
"pagination": {}
"pagination": {"totalCount" : 0, "pageSize" : 0, "totalPages" : 0, "currentPage" : 0}

Index Based Pagination

Most endpoints in BrAPI use page index numbers to determine how to get the next page of data. Page numbers start at zero, so the first page will be page 0 (zero). When appropriate, a request will have two parameters for paging: page and pageSize. page allows the client to request a specific page number. Page numbers start at zero, so the first page will be ?page=0. pageSize allows the client to control the maximum number of records that will be returned per page. A typical paged response will have four fields: currentPage, pageSize, totalCount and totalPages. currentPage is the number for the current returned page. In most cases, this will be a confirmation of the requested page parameter. If page is not populated in the request, then currentPage should have the default value zero, indicating the first page of data was retrieved by default. pageSize indicates the number of records in the current response page. totalCount indicates the total number of records available in the unpaged super set. totalPages indicates the total number of pages available and can be calculated with the formula totalCount / requested pageSize.

Token Based Pagination

A small subset of BrAPI endpoints use a nextPageToken instead of an page number. These endpoints are related to Genotyping and the alternative paging scheme is better suited to handle larger data sets. These endpoints are clearly marked as different from the norm.

A traditional index based paging system uses the page number and the page size to calculate a number of records somewhere in the middle of a dataset. When used in a SQL query, this typically takes the form “… OFFSET (page*pageSize) LIMIT (pageSize)”. OFFSET is used to count records to get to the middle of a table, and unfortunately, OFFSET is not a very efficient operation for large data tables. Token based pagination fixes this inefficiency by using a WHERE clause instead and allowing a query to jump directly to the middle of a table instead of counting records to reach the middle. With token based pagination, the SQL typically looks like this: “… WHERE key > (pageToken) ORDER BY key LIMIT (pageSize)”.

General Notes

  • When a request is for either the first page of data or the last page of data, there is a corner case where prevPageToken or nextPageToken is undefined. In these cases, the response should either exclude the undefined field or set the value to the JSON null value.

Implementation Options

Option #1: Use Index numbers as Tokens

Token based pagination is designed to provide efficiency when querying large data sets, but that efficiency is not required in all cases. It is perfectly acceptable to assume that the page token values are integers and continue to use tradition index based paging algorithms and the OFFSET clause. In this case, if a request is made with “pageToken=2”, then the response would be

{…
“currentPageToken”:”2”,
“nextPageToken”:”3”,
“prevPageToken”:”1”
…}

On the server side, this is exactly the same as index based pagination, except the parameter names have changed, and the server is providing an automatically incremented value to the next page number.

Option #2: Use Primary Key as Token

When doing Token based pagination, the first request will only have a pageSize and no pageToken. The server should respond with the first page of data, ordered by the primary key of the table. The primary key value of the last record of this first page should be returned as “nextPageToken”. The client application can now begin to make a sequential series of calls to the server, using the “nextPageToken” to get each next page of results. The data is ordered by the primary key, and on each request the client provides the value of the primary key where the previous page ended, so a SQL query can very quickly find the position where it left off and collect the next page of data.

One down side to this approach is that you must move through the data sequentially, you cannot jump to an arbitrary page of data. Token based pagination is better for use cases handling a stream of large data, it is not very good at handling use cases with a user interface where a user has the freedom to jump through the data erratically.

Option #3: Use an alternate key as Token

Using the Primary Key of a table to navigate pages can be a problem because it requires that the data is ordered by the Primary Key. This does not work for some data when there is a desire to order by another field in the table. This is doable as long as the value of the token returned is unique and always sorted consistently. A new column could be added to the table to create a sequential value in the desired sort order. Or the token returned could be a concatenation of several column values which are all sorted by simultaneously. In either solution, it is a good idea to include the primary key as part of this concatenation to guarantee the uniqueness of the token returned.