Branching strategy selection, cumulative authoring, ++ (#1445)

Author: shainaraskas

docs/_docset.yml

@@ -13,6 +13,9 @@ subs:
   serverless-short: Serverless
   ece:   "Elastic Cloud Enterprise"
   eck:   "Elastic Cloud on Kubernetes"
+  ech:  "Elastic Cloud Hosted"
+  serverless-full: "Elastic Cloud Serverless"
+  ecloud: "Elastic Cloud"
 
 features:
   primary-nav: false
@@ -31,6 +34,8 @@ toc:
       - file: on-the-web.md
       - file: move.md
       - file: redirects.md
+      - file: cumulative-docs.md
+      - file: branching-strategy.md
       - file: add-repo.md
   - folder: migration
     children:
@@ -69,6 +74,7 @@ toc:
           - file: navigation.md
           - file: extensions.md
       - file: page.md
+      - file: content-sources.md
   - folder: syntax
     children:
       - file: index.md

docs/configure/content-sources.md

@@ -0,0 +1,104 @@
+# Content sources
+
+To support multiple branching strategies for different repositories, we support the concept of a content source.
+
+Next
+:   The source for the upcoming documentation.
+
+Current
+:   The source for the active documentation.
+
+
+Our publish environments are connected to a single content source.
+
+| Publish Environment | Content Source |
+|---------------------|----------------|
+| Production          | `Current`      |
+| Staging             | `Current`      |
+| Edge                | `Next`         |
+
+This allows you as an owner of a repository to choose two different branching strategies.
+
+## Branching strategies
+
+The new documentation system supports 2 branching strategies.
+
+Continuous deployment 
+:   This is the default where a repositories `main` or `master` branch is continuously deployed to production.
+
+Tagged
+:   Allows you to 'tag' a single git reference (typically a branch) as `current` which will be used to deploy to production.
+    Allowing you to control the timing of when new documentation should go live.
+
+
+### Continuous deployment
+
+This is the default. To get started, follow our [guide](/migration/guide/index.md) to set up the new docs folder structure and CI configuration.
+
+Once setup ensure the repository is added to our `assembler.yml`  under `references`. 
+
+For example say you want to onboard `elastic/my-repository` into the production build:
+
+```yaml
+references:
+  my-repository:
+```
+
+Is equivalent to specifying.
+
+```yaml
+references:
+  my-repository:
+    next: main
+    current: main
+```
+
+% TODO we need navigation.yml docs
+Once the repository is added, its navigation still needs to be injected into to global site navigation.
+
+### Tagged
+
+If you want to have more control over the timing of when your docs go live to production. Configure the repository
+in our `assembler.yml` to have a fixed git reference (typically a branch) deploy the `current` content source to production.
+
+```yaml
+references:
+  my-other-repository:
+    next: main
+    current: 9.0
+```
+
+:::{note}
+In order for `9.0` to be onboarded it needs to first follow our [migration guide](/migration/guide/index.md) and have our documentation CI integration setup.
+Our CI integration checks will block until `current` is successfully configured
+:::
+
+#### CI configuration
+
+To ensure repositories that use the [tagged branching strategy](#tagged) can be onboarded correctly, our CI integration needs to have appropriate `push`
+ branch triggers.
+
+```yml
+name: docs-build
+
+on:
+  push:
+    branches:
+      - main
+      - '\d+.\d+' <1>
+  pull_request_target: ~
+  merge_group: ~
+
+jobs:
+  docs-preview:
+    uses: elastic/docs-builder/.github/workflows/preview-build.yml@main
+    with:
+      path-pattern: docs/**
+    permissions:
+      deployments: write
+      id-token: write
+      contents: read
+      pull-requests: write
+```
+
+1. Ensure version branches are built and publish their links ahead of time.

docs/contribute/_snippets/docs-intro.md

@@ -0,0 +1,5 @@
+elastic.co/docs uses our new build system, also known as "Docs V3", which uses an extended version of markdown as its markup language. Refer to our [syntax guide](/syntax/index.md) to learn more.
+
+Docs for {{stack}} 9.x, {{ece}} 4.x, and {{eck}} 3.x can all be found on this site. All unversioned products, such as {{ech}} and {{serverless-full}}, are also documented on elastic.co/docs.
+
+This documentation is **cumulative**. This means that a new set of docs is not published for every minor release. Instead, each page stays valid over time and incorporates version-specific changes directly within the content. [Learn how to write cumulative documentation](/contribute/cumulative-docs.md).

docs/contribute/_snippets/guide-intro.md

@@ -0,0 +1,3 @@
+To contribute to elastic.co/guide, you must work with our [legacy documentation build system](https://github.com/elastic/docs). This system uses ASCIIDoc as its markup language.
+
+Docs for {{stack}} 8.x, {{ece}} 3.x, and {{eck}} 2.x can all be found on this site.

docs/contribute/_snippets/tag-processing.md

@@ -0,0 +1,21 @@
+`applies_to` tags are rendered as badges in the documentation output. They reproduce the "key + lifecycle status + version" indicated in the content sources.
+
+Specifically for versioned products, badges will display differently when the `applies_to` key specifies a product version that has not been released to our customers yet.
+
+* `Planned` (if the lifecycle is preview, beta, or ga)
+  
+  Example: {applies_to}`stack: ga 99.99`
+* `Deprecation planned` (if the lifecycle is deprecated)
+  
+  Example: {applies_to}`stack: deprecated 99.99`
+* `Removal planned` (if the lifecycle is removed) 
+
+  Example: {applies_to}`stack: removed 99.99`
+
+This is computed at build time (there is a docs build every 30 minutes). The documentation team tracks and maintains released versions for these products centrally in [versions.yml](https://github.com/elastic/docs-builder/blob/main/src/Elastic.Documentation.Configuration/versions.yml).
+
+When multiple lifecycle statuses and versions are specified in the sources, several badges are shown.
+
+:::{note}
+Visuals and wording in the output documentation are subject to changes and optimizations.
+:::

docs/contribute/_snippets/tagged-warning.md

@@ -0,0 +1,7 @@
+:::{warning}
+Some repositories use a [tagged branching strategy](/contribute/branching-strategy.md), which means that their docs are published from a branch that is not `main` or `master`. In these cases, documentation changes need to be made to `main` or `master`, and then backported to the relevant branches.
+
+For detailed backporting guidance, refer to the example in [Choose the docs branching strategy for a repository](/contribute/branching-strategy.md#workflow-2-tagged).
+
+To determine the published branches for a repository, find the repository in [assembler.yml](https://github.com/elastic/docs-builder/blob/main/src/tooling/docs-assembler/assembler.yml).
+:::

docs/contribute/_snippets/two-systems.md

@@ -0,0 +1,7 @@
+Because the new documentation site only hosts documentation for newer releases, the way that you contribute depends on the version that you want to document.
+
+For a list of versions covered on [elastic.co/docs](https://www.elastic.co/docs), refer to [Find docs for your product version](https://www.elastic.co/docs/get-started/versioning-availability#find-docs-for-your-product-version). Versions prior to the versions listed on that page are documented on our legacy system, [elastic.co/guide](https://www.elastic.co/guide). 
+
+:::{tip}
+Unversioned products, such as {{ech}} and {{serverless-full}}, are documented on [elastic.co/docs](https://www.elastic.co/docs).
+:::

docs/contribute/add-repo.md

@@ -54,6 +54,10 @@ references:
   yadda-docs:
 ```
 
+:::{tip}
+In this file, you can optionally specify custom branches to deploy docs from, depending on your preferred [branching strategy](branching-strategy.md). You might want to change your branching strategy so you can have more control over when content added for a specific release is published.
+:::
+
 Then, edit the [navigation.yml](https://github.com/elastic/docs-builder/blob/main/src/tooling/docs-assembler/navigation.yml) file to add the repository to the navigation.
 
 For example, to add the `elastic/yadda-docs` repository under **Reference**:

docs/contribute/branching-strategy.md

@@ -0,0 +1,136 @@
+---
+navigation_title: Choose a branching strategy
+---
+
+# Choose the docs branching strategy for a repository
+
+With Docs V3 (elastic.co/docs), a single branch is published per repository. This branch is set to `main` by default. This is known as the continuous deployment branching strategy. However, it is possible to instead publish a different branch, also known as the tagged branching strategy. 
+
+On this page, you'll learn how to choose the right branching strategy for your repository, and how to change the branching strategy. You'll also learn about the workflows for working with each branching strategy.
+
+## Why is `main` the default publication branch?
+
+The main reasons for this choice are:
+
+* With Docs V3, there is no longer a different version of each page for each minor release. Instead, the same page [covers all versions](cumulative-docs.md), and any changes are indicated throughout the content.  
+* More and more products are released from `main` branches, making these branches the most up to date at any given time. This is especially true for {{serverless-full}} and {{ecloud}}.
+
+ 
+## Why would we want to publish a different branch instead?
+
+Publishing from the main branch isn’t the best option for all repositories.
+
+* `main` can contain code and docs for unreleased versions that we don’t want to publish yet.  
+* The versioning scheme and release cadence of the product associated with a repository can vary, and it can be inconsistent to have the docs associated with a certain version live in a different branch than the code.
+
+If you choose this publication model for your repository AND that repository includes {{serverless-short}} or {{ecloud}} documentation, you will need to make sure that {{serverless-short}}- and {{ecloud}}-related changes are also backported to the branch that is publishing to the public docs site.
+
+You **don't** need to change your branching strategy to enable writing docs about future versions. Review the [continuous deployment workflow](#workflow-1-default-continuous-deployment) and [](cumulative-docs.md) to learn more.
+
+Note that regardless of the publication branch that is set, the documentation must still flag all changes introduced so far since the last major release. This is NOT an alternative to [writing docs cumulatively](cumulative-docs.md).
+
+## How to change the published branch
+
+Choosing to switch between publishing docs from `main` and publishing docs from a version branch is a long-term decision. This decision impacts all docs for an entire repository. Reach out to the docs team to discuss the change.
+
+For more information, refer to [](/configure/content-sources.md).
+
+After it has been established that a repository should publish from a version branch rather than `main`:
+
+1. [Add new triggers to the `docs-build` CI integration](/configure/content-sources.md#ci-configuration). Merge these changes to `main` or `master` and the intended version branches.
+2. Open a PR to trigger the CI integration and confirm that the docs build.
+3. Open a PR updating the [docs assembler file](https://github.com/elastic/docs-builder/blob/main/src/tooling/docs-assembler/assembler.yml):  
+   * Specify which is the `current` branch for the repository. This branch is the branch from which docs are deployed to production at [elastic.co/docs](http://elastic.co/docs).  
+   * Specify which is the `next` branch for the repository. The branch defined as `next` publishes docs internally to [staging-website.elastic.co/docs](http://staging-website.elastic.co/docs)  
+     * Setting this branch to the next version branch in line is a good practice to preview docs change for an upcoming version.  
+     * Otherwise, keeping it set to `main` is also an option since this is where the content is initially developed and merged. This is the default.
+4. In the assembler PR, add the `ci` label. After CI runs, confirm that the intended version branches are publishing to the link service. When links are being published as intended, they can be found at the following URL, where `repo` is your repo name and `branch` is your newly configured branch:
+
+  ```text
+  elastic-docs-link-index.s3.us-east-2.amazonaws.com/elastic/<repo>/<branch>/links.json
+  ```
+5. Rerun the `validate-assembler` check on the PR.
+6. After checks pass and the docs engineering team approves, you can merge the PR.
+
+After these steps are completed, the docs engineering team needs to release a new version of our build tool to complete the process. This process will be decoupled in a future release. After a new version is released, the switch is complete and the production documentation reflects the specified current branch.
+
+### Update the release process
+
+When you publish from specific version branches, you need to bump the version branch as part of the release process.
+
+Add an action as part of that repo’s release process for the release manager to update this same assembler file and bump the `current` branch with each release, as appropriate. The `next` branch also needs to be bumped if it is not set to `main`. 
+
+When these releases happen, create a PR against the [assembler file](https://github.com/elastic/docs-builder/blob/main/src/tooling/docs-assembler/assembler.yml) that defines the new `current` branch, to merge on release day.
+
+:::{tip}
+Regardless of the branching strategy, you also need to update the current version in [versions.yml](https://github.com/elastic/docs-builder/blob/main/src/Elastic.Documentation.Configuration/versions.yml) as part of your release process. This version number is used in documentation variables and drives our [dynamic version badge logic](/contribute/cumulative-docs.md#how-do-these-tags-behave-in-the-output).
+:::
+
+
+## Workflow 1 (default): Continuous deployment
+
+Learn how to make updates in the continuous deployment branching strategy, where the repo is publishing docs from `main`.
+
+### Where to make docs changes [make-changes-cd]
+
+Initiate the changes by opening a PR against the `main` branch of the repo.
+
+### How to write those changes [write-changes-cd]
+
+In elastic.co/docs (Docs V3), we [write docs cumulatively](cumulative-docs.md) regardless of the branching strategy selected.
+
+### Merging and backporting [merge-backport-cd]
+
+When a repo publishes docs from its `main` branch, any merged changes are published within 30 minutes. It is then very important to consider the timing of the merge depending on the documented product:
+
+| | Case | Approach |
+| --- | --- | --- |
+| 1 | You are documenting changes for an unversioned product (typically Serverless or Elastic Cloud), and the changes should only go live when the corresponding code or feature is available to users. | The PR should be merged on or after the release date of the feature. |
+| 2 | You are documenting changes for a versioned product (any Stack components, ECE, ECK, etc.). | You have the choice between merging the PR as soon as it is approved, or merging it only on release day.<br><br>We have an automatic mechanism in place as part of the [cumulative docs strategy](cumulative-docs.md) that will show any changes published before its associated code or feature is available as `Planned`. |
+| 3 | You are documenting changes that apply to both versioned and unversioned products (typically a change that is being released for both Serverless and an upcoming Stack release). | The PR should only be merged on or after the release date of the feature in Serverless.<br><br>For versioned products, we have an automatic mechanism in place as part of the [cumulative docs strategy](cumulative-docs.md) that will show any changes published before its associated code or feature is available as `Planned`. |
+
+When a repo is publishing docs from its `main` branch, no backporting is needed.
+
+:::{tip}
+If you don’t want to hold on too many PRs to publish on release day, merge them to a feature branch, so you only have to merge this feature branch to `main` on release day.
+:::
+
+## Workflow 2: Tagged
+
+Learn how to make updates in the continuous deployment branching strategy, where the repo is publishing docs from a specific version branch. 
+
+### Where to make docs changes [make-changes-td]
+
+Initiate the changes by opening a PR against the `main` branch of the repo, and backport the changes to the relevant version branches as detailed below.
+
+### How to write those changes [write-changes-td]
+
+In elastic.co/docs (Docs V3), we [write docs cumulatively](cumulative-docs.md) regardless of the branching strategy selected.
+
+### Merging and backporting [merge-backport-td]
+
+When a repo publishes docs from a version branch, there is no timing constraint to merge the initial PR against the `main` branch. If `main` is set as your `next` branch, then the docs changes become visible on the internal staging docs site at [staging-website.elastic.co/docs](http://staging-website.elastic.co/docs). Otherwise, the docs changes become visible on the internal staging docs site when you backport to your `next` branch.
+
+The changes must then be backported to their relevant version branches, and no further back than the branch set as `current` for the documentation publication system:
+
+| | Case | Approach |
+| --- | --- | --- |
+| 1 | You are documenting changes for an unversioned product (typically Serverless or Elastic Cloud), the changes should go live when the corresponding code or feature is available to users. | The PR should be backported to the docs `current` branch, and any intermediate version branches that already exist between `current` and `main`. Merge the backport PR for `current` only when you’re sure the corresponding feature is released. |
+| 2 | You are documenting changes for a versioned product (any Stack components, ECE, ECK, etc.). | Backport the PR to the relevant version branch and to any intermediate version branch that already exists. The changes will go live whenever these branches become the `current` docs branch.<br><br>We have an automatic mechanism in place as part of the [cumulative docs strategy](cumulative-docs.md) that will show any changes published before its associated code or feature is available as `Planned`. |
+| 3 | You are documenting changes that apply to both versioned and unversioned products (typically a change that is being released for both Serverless and an upcoming Stack release). | The PR should be backported to the docs `current` branch, and any intermediate version branches that already exist between `current` and `main`. Merge the backport PR for `current` only when you’re sure the corresponding feature is released. <br><br>For versioned products, we have an automatic mechanism in place as part of the [cumulative docs strategy](cumulative-docs.md) that will show any changes published before its associated code or feature is available as `Planned`. |
+
+#### Example [example-td]
+
+For example, in a situation where 9.0, 9.1, and 9.2 are already released, and the 9.3 branch has already been cut:
+
+* The branch set as `current` in the [docs assembler](https://github.com/elastic/docs-builder/blob/625e75b35841be938a8df76a62deeee811ba52d4/src/tooling/docs-assembler/assembler.yml#L70) is 9.2.  
+* The branch set as `next` (where the content development first happens), is `main`.  
+* 9.4 changes are only done on the `main` branch.  
+* 9.3 changes are done on the `main` branch and backported to the 9.3 branch.  
+* 9.1 changes are done on the `main` branch and backported to the 9.3 and 9.2 branches. Since 9.2 is the current docs branch, no need to go further.  
+* Serverless changes are done on the `main` branch. They are backported to the `current` docs branch and any intermediate branches. In this case: 9.3 and 9.2.  
+* Changes not specific to a version are done on the `main` branch and backported to all branches down to the `current` docs branch. In this case: 9.3 and 9.2.
+
+:::{note}
+While you *can* backport to versions prior to the `current` version when applicable to maintain parity between the code and the docs on a given branch, that content will not be used in the current state of the docs.
+:::