Skip to content

Commit 8a8dc41

Browse files
cloudant-sdks-automationricellis
cloudant-sdks-automation
authored and
ricellis
committed
docs(generated): pagination feature readme
Generated SDK source code using: - Generator version 3.106.0 - Specification version 1.0.0-dev0.1.28 - Automation (cloudant-sdks) version 6519883
1 parent c94257c commit 8a8dc41

1 file changed

Lines changed: 315 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,23 @@ to avoid surprises.
6262
* [Process continuous changes](#process-continuous-changes)
6363
* [Process one-off changes](#process-one-off-changes)
6464
- [Stopping the changes follower](#stopping-the-changes-follower)
65+
* [Pagination (beta)](#pagination-beta)
66+
+ [Introduction](#introduction-1)
67+
+ [Limitations](#limitations)
68+
+ [Capacity considerations](#capacity-considerations)
69+
+ [Available operations](#available-operations)
70+
+ [Creating a pagination](#creating-a-pagination)
71+
- [Initialize the service](#initialize-the-service)
72+
- [Set the options](#set-the-options)
73+
- [Create the pagination](#create-the-pagination)
74+
+ [Using pagination](#using-pagination)
75+
- [Stream pages](#stream-pages)
76+
- [Stream rows](#stream-rows)
77+
- [Iterate pages](#iterate-pages)
78+
- [Iterate rows](#iterate-rows)
79+
- [Pager](#pager)
80+
* [Get each page from a pager](#get-each-page-from-a-pager)
81+
* [Get all results from a pager](#get-all-results-from-a-pager)
6582
- [Questions](#questions)
6683
- [Issues](#issues)
6784
- [Versioning and LTS support](#versioning-and-lts-support)
@@ -97,6 +114,7 @@ project:
97114
- Familiar user experience with IBM Cloud SDKs.
98115
- Flexibility to use either built-in models or byte-based requests and responses for documents.
99116
- Built-in [Changes feed follower](#changes-feed-follower)
117+
- Built-in [Pagination](#pagination-beta) (beta)
100118
- HTTP2 support for higher performance connections to IBM Cloudant.
101119
- Perform requests either synchronously or asynchronously.
102120
- Instances of the client are unconditionally thread-safe.
@@ -1068,6 +1086,303 @@ public class Stop {
10681086
}
10691087
```
10701088

1089+
### Pagination (beta)
1090+
1091+
#### Introduction
1092+
1093+
The pagination feature (currently beta) accepts options for a single operation and automatically
1094+
creates the multiple requests to the server necessary to page through the results a fixed number at a time.
1095+
1096+
Pagination is a best-practice to break apart large queries into multiple server requests.
1097+
This has a number of advantages:
1098+
* Keeping requests within server imposed limits, for example
1099+
* `200` max results for text search
1100+
* `2000` max results for partitioned queries
1101+
* Fetching only the necessary data, for example
1102+
* User finds required result on first page, no need to continue fetching results
1103+
* Reducing the duration of any individual query
1104+
* Reduce risk of query timing out on the server
1105+
* Reduce risk of network request timeouts
1106+
1107+
#### Limitations
1108+
1109+
Limitations of pagination:
1110+
* Forward only, no backwards paging
1111+
* Limitations on `_all_docs` and `_design_docs` operations
1112+
* No pagination for `key` option.
1113+
There is no need to paginate as IDs are unique and this returns only a single row.
1114+
This is better achieved with a single document get request.
1115+
* No pagination for `keys` option.
1116+
* Limitations on `_view` operations
1117+
* No pagination for `key` option. Pass the same `key` as a start and end key instead.
1118+
* No pagination for `keys` option.
1119+
* Views that emit multiple identical keys (with the same or different values)
1120+
from the same document cannot paginate if those key rows with the same ID
1121+
span a page boundary.
1122+
The pagination feature detects this condition and an error occurs.
1123+
It may be possible to workaround using a different page size.
1124+
* Limitations on `_search` operations
1125+
* No pagination of grouped results.
1126+
* No pagination of faceted `counts` or `ranges` results.
1127+
1128+
#### Capacity considerations
1129+
1130+
Pagination can make many requests rapidly from a single program call.
1131+
1132+
For IBM Cloudant take care to ensure you have appropriate plan capacity
1133+
in place to avoid consuming all the permitted requests.
1134+
If there is no remaining plan allowance and retries are not enabled or insufficient
1135+
then a `429 Too Many Requests` error occurs.
1136+
1137+
#### Available operations
1138+
1139+
Pagination is available for these operations:
1140+
* Query all documents [global](https://cloud.ibm.com/apidocs/cloudant?code=java#postalldocs)
1141+
and [partitioned](https://cloud.ibm.com/apidocs/cloudant?code=java#postpartitionalldocs)
1142+
* [Global all documents examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/AllDocsPagination.java)
1143+
* [Partitioned all documents examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/PartitionAllDocsPagination.java)
1144+
* Query all [design documents](https://cloud.ibm.com/apidocs/cloudant?code=java#postdesigndocs)
1145+
* [Design documents examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/DesignDocsPagination.java)
1146+
* Query with selector syntax [global](https://cloud.ibm.com/apidocs/cloudant?code=java#postfind)
1147+
and [partitioned](https://cloud.ibm.com/apidocs/cloudant?code=java#postpartitionfind)
1148+
* [Global find selector query examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/FindPagination.java)
1149+
* [Partitioned find selector query examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/PartitionFindPagination.java)
1150+
* Query a search index [global](https://cloud.ibm.com/apidocs/cloudant?code=java#postsearch)
1151+
and [partitioned](https://cloud.ibm.com/apidocs/cloudant?code=java#postpartitionsearch)
1152+
* [Global search examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/SearchPagination.java)
1153+
* [Partitioned search examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/PartitionSearchPagination.java)
1154+
* Query a MapReduce view [global](https://cloud.ibm.com/apidocs/cloudant?code=java#postview)
1155+
and [partitioned](https://cloud.ibm.com/apidocs/cloudant?code=java#postpartitionview)
1156+
* [Global view examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/ViewPagination.java)
1157+
* [Partitioned view examples](https://github.com/IBM/cloudant-java-sdk/tree/v0.10.4/modules/examples/src/main/java/features/pagination/PartitionViewPagination.java)
1158+
1159+
The examples presented in this `README` are for all documents in a partition.
1160+
The links in the list are to equivalent examples for each of the other available operations.
1161+
1162+
#### Creating a pagination
1163+
1164+
Make a new pagination from a client
1165+
and the options for the chosen operation.
1166+
Use the `limit` option to configure the page size (default and maximum `200`).
1167+
1168+
Imports required for these examples:
1169+
1170+
<details open>
1171+
<summary>Java:</summary>
1172+
1173+
```java
1174+
import java.util.List;
1175+
import com.ibm.cloud.cloudant.v1.Cloudant;
1176+
import com.ibm.cloud.cloudant.v1.model.DocsResultRow;
1177+
import com.ibm.cloud.cloudant.v1.model.PostPartitionAllDocsOptions;
1178+
import com.ibm.cloud.cloudant.features.pagination.Pager;
1179+
import com.ibm.cloud.cloudant.features.pagination.Pagination;
1180+
```
1181+
1182+
</details>
1183+
1184+
##### Initialize the service
1185+
1186+
<details open>
1187+
<summary>Java:</summary>
1188+
1189+
```java
1190+
// Initialize service
1191+
Cloudant service = Cloudant.newInstance();
1192+
```
1193+
1194+
</details>
1195+
1196+
##### Set the options
1197+
1198+
<details open>
1199+
<summary>Java:</summary>
1200+
1201+
```java
1202+
// Setup options
1203+
PostPartitionAllDocsOptions options = new PostPartitionAllDocsOptions.Builder()
1204+
.db("events") // example database name
1205+
.limit(50) // limit option sets the page size
1206+
.partitionKey("ns1HJS13AMkK") // query only this partition
1207+
.build();
1208+
```
1209+
1210+
</details>
1211+
1212+
##### Create the pagination
1213+
1214+
<details open>
1215+
<summary>Java:</summary>
1216+
1217+
```java
1218+
// Create pagination
1219+
Pagination<PostPartitionAllDocsOptions, DocsResultRow> pagination = Pagination.newPagination(service, options);
1220+
// pagination can be reused without side-effects as a factory for iterables, streams or pagers
1221+
// options are fixed at pagination creation time
1222+
```
1223+
1224+
</details>
1225+
1226+
#### Using pagination
1227+
1228+
Once you have a pagination factory there are multiple options available.
1229+
* Stream pages
1230+
* Stream rows
1231+
* Iterate pages
1232+
* Iterate rows
1233+
* Get each page from a pager
1234+
* Get all results from a pager
1235+
1236+
All the paging styles produce equivalent results and make identical page requests.
1237+
The style of paging to choose depends on the use case requirements
1238+
in particular whether to process a page at a time or a row at a time.
1239+
1240+
The pagination factory is reusable and can repeatedly produce new instances
1241+
of the same or different pagination styles for the same operation options.
1242+
1243+
Here are examples for each paging style.
1244+
1245+
##### Stream pages
1246+
1247+
Streaming pages is ideal for lazy functional processing of pages
1248+
and leveraging Java's built-in stream utilities, for example, to limit the total number of pages.
1249+
1250+
<details open>
1251+
<summary>Java:</summary>
1252+
1253+
```java
1254+
// Option: stream pages
1255+
// Ideal for lazy functional processing of pages and total page limits
1256+
pagination.pageStream() // a new stream of the pages
1257+
.limit(20) // use Java stream limit to get only the first 20 pages (different from 50 limit used for page size)
1258+
.forEach(page -> { // stream operations e.g. terminal forEach
1259+
// Do something with page
1260+
});
1261+
```
1262+
1263+
</details>
1264+
1265+
##### Stream rows
1266+
1267+
Streaming rows is ideal for lazy functional processing of each result row
1268+
and leveraging Java's built-in stream utilities, for example, to limit the total number of results.
1269+
1270+
<details open>
1271+
<summary>Java:</summary>
1272+
1273+
```java
1274+
// Option: stream rows
1275+
// Ideal for lazy functional processing of rows and total row limits
1276+
pagination.rowStream() // a new stream of the rows
1277+
.limit(1000) // use Java stream limit to cap at 1000 rows (20 page requests of 50 rows each in this example)
1278+
.forEach(row -> { // stream operations e.g. terminal forEach
1279+
// Do something with row
1280+
});
1281+
```
1282+
1283+
</details>
1284+
1285+
##### Iterate pages
1286+
1287+
Iterating pages is ideal for using an enhanced for loop to process a page at a time.
1288+
1289+
<details open>
1290+
<summary>Java:</summary>
1291+
1292+
```java
1293+
// Option: iterate pages
1294+
// Ideal for using an enhanced for loop with each page.
1295+
// The Iterable returned from pages() is reusable in that
1296+
// calling iterator() returns a new iterator each time.
1297+
// The returned iterators, however, are single use.
1298+
for (List<DocsResultRow> page : pagination.pages()) {
1299+
// Do something with page
1300+
}
1301+
```
1302+
1303+
</details>
1304+
1305+
##### Iterate rows
1306+
1307+
Iterating rows is ideal for using an enhanced for loop to process a result row at a time.
1308+
1309+
<details open>
1310+
<summary>Java:</summary>
1311+
1312+
```java
1313+
// Option: iterate rows
1314+
// Ideal for using an enhanced for loop with each row.
1315+
// The Iterable returned from rows() is reusable in that
1316+
// calling iterator() returns a new iterator each time.
1317+
// The returned iterators, however, are single use.
1318+
for (DocsResultRow row : pagination.rows()) {
1319+
// Do something with row
1320+
}
1321+
```
1322+
1323+
</details>
1324+
1325+
##### Pager
1326+
1327+
The pager style is similar to other [IBM Cloud SDKs](https://github.com/IBM/ibm-cloud-sdk-common?tab=readme-ov-file#pagination).
1328+
Users familiar with that style of pagination may find using them preferable
1329+
to the native language style iterators.
1330+
1331+
In the Cloudant SDKs these pagers are single use and traverse the complete set of pages once and only once.
1332+
After exhaustion they cannot be re-used, simply create a new one from the pagination factory if needed.
1333+
1334+
Pagers are only valid for one of either page at a time or getting all results.
1335+
For example, calling for the next page then calling for all results causes an error.
1336+
1337+
###### Get each page from a pager
1338+
1339+
This is useful for calling to retrieve one page at a time, for example,
1340+
in a user interface with a "next page" interaction.
1341+
1342+
If calling for the next page errors, it is valid to call for the next page again
1343+
to continue paging.
1344+
1345+
<details open>
1346+
<summary>Java:</summary>
1347+
1348+
```java
1349+
// Option: use pager next page
1350+
// For retrieving one page at a time with a method call.
1351+
Pager<DocsResultRow> pagePager = pagination.pager();
1352+
if (pagePager.hasNext()) {
1353+
List<DocsResultRow> page = pagePager.getNext();
1354+
// Do something with page
1355+
}
1356+
```
1357+
1358+
</details>
1359+
1360+
###### Get all results from a pager
1361+
1362+
This is useful to retrieve all results in a single call.
1363+
However, this approach requires sufficient memory for the entire collection of results.
1364+
So although it may be convenient for small result sets generally prefer iterating pages
1365+
or rows with the other paging styles, especially for large result sets.
1366+
1367+
If calling for all the results errors, then calling for all the results again restarts the pagination.
1368+
1369+
<details open>
1370+
<summary>Java:</summary>
1371+
1372+
```java
1373+
// Option: use pager all results
1374+
// For retrieving all result rows in a single list
1375+
// Note: all result rows may be very large!
1376+
// Preferably use streams/iterables instead of getAll for memory efficiency with large result sets.
1377+
Pager<DocsResultRow> allPager = pagination.pager();
1378+
List<DocsResultRow> allRows = allPager.getAll();
1379+
for (DocsResultRow row : allRows) {
1380+
// Do something with row
1381+
}
1382+
```
1383+
1384+
</details>
1385+
10711386
## Questions
10721387

10731388
If you are having difficulties using this SDK or have a question about the

0 commit comments

Comments
 (0)