Long polling significantly simplifies the client algorithm as the client does not need to implement different handling if there are further items in the feed available. There is no sleep step involved in normal processing. Long polling minimizes latency when new items arrive in the feed.
On the downside, long polling has higher demands on the server side, as the requests connections must be kept open for some seconds. This should not be a problem for a few clients (< 100) and should be feasible for many parallel clients with reactive implementations.
In a simple implementation, the server queries the database for new items in a short interval (such as 50 millis) until there are new items or a timeout occurs (> 1 seconds, < 10 seconds recommended).
We use dynamic offset querying of the next feed items using the position as offset.
Discussed alternatives:
- Static linked pages, but higher traffic while polling last page and to limited for filtering.
Every feed item contains a next link to fetch subsequent items.
While it would be sufficient (and feeling more natural) to have a next link on page level, it makes each feed item self-contained. In case of processing or appliction error in a subsequent item on the same page, the client can make the next fetch starting from the last successfully processed item's next link. This significantly simplifies exception and retry handling on the client side.
We use plain application/json as media type and return a simple array of items.
Discussed alternatives:
- JSON:API, reasonable, but very nested and complex for embedded data (the
includedarray). Specialapplication/vnd.api+jsonmedia type. - HAL, but to limited and not widly supported.
- JSON-LD, but not widly adopted.
- Collection+JSON, but unnecessary complex nesting.
The page limit is defined by server and should be set sized reasonable based on the payload size. The actual page limit may vary for different pages.
This this is simple, protects the server from too large data loads and DoS, enables caching and enables the server to choose optimized storage systems.
Feed endpoints may choose to support a limit query parameter, e.g. for low bandwidth clients.
The server may ignore or override the limit. The server should always bound the upper limit.
We decided against a home document (such as JSON Home).
REST feeds is designed to be as simple as possible.
The feed starts with the plain feed URL and the client just needs to follow next links.
When the feed endpoint changes, the old endpoint should return a 302 redirect to the new endpoint.