[8.x] [skip CI] [ftr] add migration steps for deployment-agnostic tes…

…ts (#202913) (#208179)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[skip CI] [ftr] add migration steps for deployment-agnostic tests
(#202913)](#202913)

<!--- Backport version: 9.6.4 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-12-04T15:25:09Z","message":"[skip
CI] [ftr] add migration steps for deployment-agnostic tests
(#202913)\n\n## Summary\r\n\r\nThis PR adds example of tests, that
doesn't meet deployment-agnostic\r\ncriteria. I also added steps
explaining how to migrate existing tests
to\r\nDA.","sha":"72c906b25b8c8ebe430396a610ade0ad7f1d4baa","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:version","v8.17.0","v8.18.0"],"title":"[skip
CI] [ftr] add migration steps for deployment-agnostic
tests","number":202913,"url":"https://github.com/elastic/kibana/pull/202913","mergeCommit":{"message":"[skip
CI] [ftr] add migration steps for deployment-agnostic tests
(#202913)\n\n## Summary\r\n\r\nThis PR adds example of tests, that
doesn't meet deployment-agnostic\r\ncriteria. I also added steps
explaining how to migrate existing tests
to\r\nDA.","sha":"72c906b25b8c8ebe430396a610ade0ad7f1d4baa"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/202913","number":202913,"mergeCommit":{"message":"[skip
CI] [ftr] add migration steps for deployment-agnostic tests
(#202913)\n\n## Summary\r\n\r\nThis PR adds example of tests, that
doesn't meet deployment-agnostic\r\ncriteria. I also added steps
explaining how to migrate existing tests
to\r\nDA.","sha":"72c906b25b8c8ebe430396a610ade0ad7f1d4baa"}},{"branch":"8.17","label":"v8.17.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/202963","number":202963,"state":"MERGED","mergeCommit":{"sha":"5186c111d15577b309fe7192d970d531d1f170af","message":"[8.17]
[skip CI] [ftr] add migration steps for deployment-agnostic tests
(#202913) (#202963)\n\n# Backport\n\nThis will backport the following
commits from `main` to `8.17`:\n- [[skip CI] [ftr] add migration steps
for deployment-agnostic
tests\n(#202913)](https://github.com/elastic/kibana/pull/202913)\n\n<!---
Backport version: 9.4.3 -->\n\n### Questions ?\nPlease refer to the
[Backport
tool\ndocumentation](https://github.com/sqren/backport)\n\n<!--BACKPORT
[{\"author\":{\"name\":\"Dzmitry\nLemechko\",\"email\":\"[email protected]\"},\"sourceCommit\":{\"committedDate\":\"2024-12-04T15:25:09Z\",\"message\":\"[skip\nCI]
[ftr] add migration steps for deployment-agnostic
tests\n(#202913)\\n\\n## Summary\\r\\n\\r\\nThis PR adds example of
tests, that\ndoesn't meet deployment-agnostic\\r\\ncriteria. I also
added steps\nexplaining how to migrate existing
tests\nto\\r\\nDA.\",\"sha\":\"72c906b25b8c8ebe430396a610ade0ad7f1d4baa\",\"branchLabelMapping\":{\"^v9.0.0$\":\"main\",\"^v8.18.0$\":\"8.x\",\"^v(\\\\d+).(\\\\d+).\\\\d+$\":\"$1.$2\"}},\"sourcePullRequest\":{\"labels\":[\"release_note:skip\",\"v9.0.0\",\"backport:version\",\"v8.17.0\"],\"title\":\"[skip\nCI]
[ftr] add migration steps for
deployment-agnostic\ntests\",\"number\":202913,\"url\":\"https://github.com/elastic/kibana/pull/202913\",\"mergeCommit\":{\"message\":\"[skip\nCI]
[ftr] add migration steps for deployment-agnostic
tests\n(#202913)\\n\\n## Summary\\r\\n\\r\\nThis PR adds example of
tests, that\ndoesn't meet deployment-agnostic\\r\\ncriteria. I also
added steps\nexplaining how to migrate existing
tests\nto\\r\\nDA.\",\"sha\":\"72c906b25b8c8ebe430396a610ade0ad7f1d4baa\"}},\"sourceBranch\":\"main\",\"suggestedTargetBranches\":[\"8.17\"],\"targetPullRequestStates\":[{\"branch\":\"main\",\"label\":\"v9.0.0\",\"branchLabelMappingKey\":\"^v9.0.0$\",\"isSourceBranch\":true,\"state\":\"MERGED\",\"url\":\"https://github.com/elastic/kibana/pull/202913\",\"number\":202913,\"mergeCommit\":{\"message\":\"[skip\nCI]
[ftr] add migration steps for deployment-agnostic
tests\n(#202913)\\n\\n## Summary\\r\\n\\r\\nThis PR adds example of
tests, that\ndoesn't meet deployment-agnostic\\r\\ncriteria. I also
added steps\nexplaining how to migrate existing
tests\nto\\r\\nDA.\",\"sha\":\"72c906b25b8c8ebe430396a610ade0ad7f1d4baa\"}},{\"branch\":\"8.17\",\"label\":\"v8.17.0\",\"branchLabelMappingKey\":\"^v(\\\\d+).(\\\\d+).\\\\d+$\",\"isSourceBranch\":false,\"state\":\"NOT_CREATED\"}]}]\nBACKPORT-->\n\nCo-authored-by:
Dzmitry Lemechko
<[email protected]>"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <[email protected]>

x-pack/test/api_integration/deployment_agnostic/README.md

@@ -9,6 +9,12 @@ A deployment-agnostic API integration test is a test suite that fulfills the fol
 
 A deployment-agnostic test should be loaded in stateful and at least 1 serverless FTR config files.
 
+## Tests, that are not deployment-agnostic:
+- tests verifying Kibana behavior under a basic license.
+- tests dependent on ES/Kibana server arguments, that are not set in Elastic Cloud
+- tests requiring a custom plugin to be loaded specifically for testing purposes.
+- tests dependent on varying user privileges between serverless and stateful environments.
+
 ## Tests Design Requirements
 A deployment-agnostic test is contained within a single test file and always utilizes the [DeploymentAgnosticFtrProviderContext](https://github.com/elastic/kibana/blob/main/x-pack/test/api_integration/deployment_agnostic/ftr_provider_context.d.ts) to load compatible FTR services. A compatible FTR service must support:
 
@@ -243,3 +249,15 @@ node scripts/functional_test_runner --config x-pack/test/api_integration/deploym
 Since deployment-agnostic tests are designed to run both locally and on MKI/Cloud, we believe no extra tagging is required. If a test is not working on MKI/Cloud or both, there is most likely an issue with the FTR service or the configuration file it uses.
 
 When a test fails on CI, automation will apply `.skip` to the top-level describe block. This means the test will be skipped in **both serverless and stateful environments**. If a test is unstable in a specific environment only, it is probably a sign that the test is not truly deployment-agnostic.
+
+## Migrating existing tests
+If your tests align with the outlined criteria and requirements, you can migrate them to deployment-agnostic by following these steps:
+
+1. Move your tests to the `x-pack/test/api_integration/deployment_agnostic/apis/<plugin>` directory.
+2. Update each test file to use the `DeploymentAgnosticFtrProviderContext` context and load the required FTR services it provides.
+3. Ensure the `roleScopedSupertest` or `samlAuth` service is used instead for `supertest` for authentication and test API calls.
+4. Remove all usage of the `supertest` service. It is authenticated as system index superuser and often causes test failures on Cloud, where priveleges are more strict.
+5. Avoid modifying `config` files, as this could disrupt test runs in Cloud environments.
+6. Include your tests in both the platform and at least one solution index file.
+7. Execute your tests locally against a local environment.
+8. Verify your tests by running them locally against a real MKI project and an ESS deployment to ensure full compatibility.