[CDAP-21147] Make apps list call paginated in pre-upgrade and post-upgrade jobs

[adrikagupta]

We need to paginate the list apps call being made during stopAll call for programs and also in post-upgrade jobs. This is being done to avoid OOM issue in case of huge number of pipelines and to fix the upgrade operation failure from 6.10.1.2 to 6.10.1.3 as mentioned in the jira.

Testing:
Created a 6.10.1.2 instance, deployed 60 pipelines and upgraded it to 6.10.1.3 (image containing the changes)
Pre-upgrade job pod logs (page size is 25) -

2025-02-25 15:16:28,300 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@131] - Called paginated list API to schedule and workflows and got token: testPipeline31.1bc052a1-f38a-11ef-868a-96e36c277f10
2025-02-25 15:16:28,315 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline1.95bb3d83-f389-11ef-95dc-96e36c277f10
2025-02-25 15:16:28,365 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline10.c107d5cc-f389-11ef-8a38-96e36c277f10
2025-02-25 15:16:28,396 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline11.c5ec46cd-f389-11ef-82bc-96e36c277f10
2025-02-25 15:16:28,424 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline12.caa3181e-f389-11ef-801f-96e36c277f10
2025-02-25 15:16:28,451 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline13.cec327af-f389-11ef-a659-96e36c277f10
2025-02-25 15:16:28,480 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline14.d2fba140-f389-11ef-b9b0-96e36c277f10
2025-02-25 15:16:28,508 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline15.d73d69a1-f389-11ef-8eed-96e36c277f10
2025-02-25 15:16:28,536 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline16.db75bc22-f389-11ef-b8dd-96e36c277f10
2025-02-25 15:16:28,568 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline17.dfa07a13-f389-11ef-87eb-96e36c277f10
2025-02-25 15:16:28,595 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline18.e3d7bb24-f389-11ef-9c70-96e36c277f10
2025-02-25 15:16:28,623 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline19.e80f9875-f389-11ef-a866-96e36c277f10
2025-02-25 15:16:28,649 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline2.9d745254-f389-11ef-aee0-96e36c277f10
2025-02-25 15:16:28,677 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline20.ec461636-f389-11ef-a1fc-96e36c277f10
2025-02-25 15:16:28,705 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline21.f06f2677-f389-11ef-a303-96e36c277f10
2025-02-25 15:16:28,731 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline22.f4ae56c8-f389-11ef-b894-96e36c277f10
2025-02-25 15:16:28,758 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline23.f8de44d9-f389-11ef-9177-96e36c277f10
2025-02-25 15:16:28,786 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline24.fd0a625a-f389-11ef-8fee-96e36c277f10
2025-02-25 15:16:28,813 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline25.014a7d0b-f38a-11ef-8226-96e36c277f10
2025-02-25 15:16:28,841 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline26.058bd03c-f38a-11ef-9bea-96e36c277f10
2025-02-25 15:16:28,869 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline27.0a48e31d-f38a-11ef-85ad-96e36c277f10
2025-02-25 15:16:28,896 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline28.0e89e82e-f38a-11ef-8a91-96e36c277f10
2025-02-25 15:16:28,924 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline29.12c8ca5f-f38a-11ef-a34a-96e36c277f10
2025-02-25 15:16:28,951 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline3.a1a6b165-f389-11ef-9db8-96e36c277f10
2025-02-25 15:16:28,977 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline30.16e97630-f38a-11ef-bafc-96e36c277f10
2025-02-25 15:16:29,003 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline31.1bc052a1-f38a-11ef-868a-96e36c277f10
2025-02-25 15:16:29,135 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@131] - Called paginated list API to schedule and workflows and got token: testPipeline54.7f6b0088-f38a-11ef-b3a5-96e36c277f10
2025-02-25 15:16:29,136 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline32.1fff0dc2-f38a-11ef-add7-96e36c277f10
2025-02-25 15:16:29,164 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline33.242b2b43-f38a-11ef-b5a8-96e36c277f10
2025-02-25 15:16:29,191 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline34.28eecdd4-f38a-11ef-bc4e-96e36c277f10
2025-02-25 15:16:29,220 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline35.2d335555-f38a-11ef-bf77-96e36c277f10
2025-02-25 15:16:29,246 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline36.31f80956-f38a-11ef-b0a8-96e36c277f10
2025-02-25 15:16:29,273 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline37.362a6867-f38a-11ef-a0d6-96e36c277f10
2025-02-25 15:16:29,299 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline38.3a66b288-f38a-11ef-b3e3-96e36c277f10
2025-02-25 15:16:29,326 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline39.3e94a4c9-f38a-11ef-b4f9-96e36c277f10
2025-02-25 15:16:29,352 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline4.a5f523f6-f389-11ef-b710-96e36c277f10
2025-02-25 15:16:29,379 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline40.42c5080a-f38a-11ef-b606-96e36c277f10
2025-02-25 15:16:29,405 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline41.46fff29b-f38a-11ef-a914-96e36c277f10
2025-02-25 15:16:29,431 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline42.4b3a67fc-f38a-11ef-9306-96e36c277f10
2025-02-25 15:16:29,457 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline43.4f70e5bd-f38a-11ef-b323-96e36c277f10
2025-02-25 15:16:29,483 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline44.543376de-f38a-11ef-8c72-96e36c277f10
2025-02-25 15:16:29,509 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline45.58622c6f-f38a-11ef-84e5-96e36c277f10
2025-02-25 15:16:29,534 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline46.5c9a7ef0-f38a-11ef-9046-96e36c277f10
2025-02-25 15:16:29,560 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline47.6171d091-f38a-11ef-af4b-96e36c277f10
2025-02-25 15:16:29,585 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline48.663b3f82-f38a-11ef-bf8e-96e36c277f10
2025-02-25 15:16:29,610 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline49.6a62a213-f38a-11ef-b7e1-96e36c277f10
2025-02-25 15:16:29,636 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline5.aad4da07-f389-11ef-94b4-96e36c277f10
2025-02-25 15:16:29,662 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline50.6e9d8ca4-f38a-11ef-8bc1-96e36c277f10
2025-02-25 15:16:29,727 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline51.72dbf9a5-f38a-11ef-9100-96e36c277f10
2025-02-25 15:16:29,754 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline52.76fec856-f38a-11ef-a003-96e36c277f10
2025-02-25 15:16:29,780 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline53.7b340d97-f38a-11ef-86c9-96e36c277f10
2025-02-25 15:16:29,806 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline54.7f6b0088-f38a-11ef-b3a5-96e36c277f10
2025-02-25 15:16:29,878 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@131] - Called paginated list API to schedule and workflows and got token: null
2025-02-25 15:16:29,879 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline55.843b7459-f38a-11ef-963a-96e36c277f10
2025-02-25 15:16:29,905 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline56.88682e1a-f38a-11ef-a5a4-96e36c277f10
2025-02-25 15:16:29,931 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline57.8ca7faab-f38a-11ef-98dc-96e36c277f10
2025-02-25 15:16:29,957 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline58.916841dc-f38a-11ef-acf3-96e36c277f10
2025-02-25 15:16:29,983 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline59.95a48bfd-f38a-11ef-86d3-96e36c277f10
2025-02-25 15:16:30,009 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline6.af82d1b8-f389-11ef-98dd-96e36c277f10
2025-02-25 15:16:30,034 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline60.99de3e0e-f38a-11ef-8fd3-96e36c277f10
2025-02-25 15:16:30,059 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline7.b3c5d299-f389-11ef-914e-96e36c277f10
2025-02-25 15:16:30,085 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline8.b8157daa-f389-11ef-b85b-96e36c277f10
2025-02-25 15:16:30,110 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:default.testPipeline9.bcc745eb-f389-11ef-9ea6-96e36c277f10
2025-02-25 15:16:30,262 - DEBUG [main:i.c.c.c.ProgramClient@285] - Called paginated list API to stop programs and got token: testPipeline31.1bc052a1-f38a-11ef-868a-96e36c277f10
2025-02-25 15:16:30,820 - DEBUG [main:i.c.c.c.ProgramClient@285] - Called paginated list API to stop programs and got token: testPipeline54.7f6b0088-f38a-11ef-b3a5-96e36c277f10
2025-02-25 15:16:31,329 - DEBUG [main:i.c.c.c.ProgramClient@285] - Called paginated list API to stop programs and got token: null
2025-02-25 15:16:31,516 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@131] - Called paginated list API to schedule and workflows and got token: null
2025-02-25 15:16:31,516 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:system.dataprep.-SNAPSHOT
2025-02-25 15:16:31,521 - DEBUG [main:i.c.c.m.u.UpgradeJobMain@141] - Trying to stop schedule and workflows for application application:system.pipeline.-SNAPSHOT
2025-02-25 15:16:31,531 - DEBUG [main:i.c.c.c.ProgramClient@285] - Called paginated list API to stop programs and got token: null

Post upgrade job pod logs -

2025-02-25 15:17:01,886 - DEBUG [main:i.c.c.m.u.PostUpgradeJobMain@129] - Called paginated list API to restart programs and got token: testPipeline31.1bc052a1-f38a-11ef-868a-96e36c277f10
2025-02-25 15:17:02,085 - DEBUG [main:i.c.c.m.u.PostUpgradeJobMain@129] - Called paginated list API to restart programs and got token: testPipeline54.7f6b0088-f38a-11ef-b3a5-96e36c277f10
2025-02-25 15:17:02,194 - DEBUG [main:i.c.c.m.u.PostUpgradeJobMain@129] - Called paginated list API to restart programs and got token: null

Made a similar change in #15871 to add paginated list apps call in pre-upgrade job

[sonarqubecloud]

Quality Gate failed

Failed conditions
43.3% Coverage on New Code (required ≥ 80%)
30.5% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud