diff --git a/doc/dev-guide/src/release-process.md b/doc/dev-guide/src/release-process.md index 0e555705eb..e53f4d5c91 100644 --- a/doc/dev-guide/src/release-process.md +++ b/doc/dev-guide/src/release-process.md @@ -1,66 +1,217 @@ -# Making a release +# Release Process -Before making a release, ensure that `rustup-init.sh` is behaving correctly, +This document describes the release process of Rustup and is split into two +sections: the first section explains the release process, while the second +section documents the steps that need to be followed to cut a release. + +- [About the Release Process](#about-the-release-process) + - [Historic Context](#historic-context) + - [Beta and Stable Releases](#beta-and-stable-releases) + - [Release Artifacts](#release-artifacts) + - [Automation](#automation) +- [How to Cut a Release](#how-to-cut-a-release) + - [1. Test `rustup-init.sh`](#1-test-rustup-initsh) + - [2. Update Version](#2-update-version) + - [3. Update Checksum](#3-update-checksum) + - [4. Sync `master` to `stable`](#4-sync-master-to-stable) + - [5. Test the `beta` Release](#5-test-the-beta-release) + - [6. Prepare Blog Post for `stable` Release](#6-prepare-blog-post-for-stable-release) + - [7. Request the `stable` Release](#7-request-the-stable-release) + - [8. Tag the `stable` Release](#8-tag-the-stable-release) + +## About the Release Process + +This section explains the release process for Rustup. + +### Historic Context + +The Rustup release process has evolved over time. In the past, the process +involved more manual steps and large parts of it were executed locally. This +required careful execution and coordination to avoid accidentally overwriting +existing releases, since the tooling did not implement sanity checks that would +prevent destructive actions. + +A full step-by-step guide for the old process can be found in +the [old developer guide](https://github.com/rust-lang/rustup/blob/1cf1b5a6d80c978e0dcaabbce5f10b3861612425/doc/dev-guide/src/release-process.md). +The following summarizes how release artifacts were created and copied around. + +In the past, both beta and stable releases were started by merging a new commit +into the `stable` branch in [rust-lang/rustup]. This started a new build on +GitHub Actions that produced release +artifacts ([source](https://github.com/rust-lang/rustup/blob/1cf1b5a6d80c978e0dcaabbce5f10b3861612425/.github/workflows/ci.yaml#L144-L151)), +which were uploaded to `s3://dev-static-rust-lang-org/rustup/dist`. As new +commits were merged into `stable`, they would overwrite the artifacts from prior +builds. + +The release artifacts were then copied to the final location by running a script +on a local machine. This script would download the artifacts from the +`dev-static` bucket on S3 and upload them to the `static` bucket. The script +also generated a new manifest with the given version, and uploaded a copy of the +files to an archive directory. + +Given that the process was manual, involved copying files to the local machine, +and waiting between creating a `beta` and a `stable` release, there was a risk +of human error. When we had to update the script after four years without a +change, we [decided](https://github.com/rust-lang/rustup/pull/3819) to redesign +and automate the release process. + +### Beta and Stable Releases + +Rustup can be released to two different environments: `beta` and `stable`. The +main difference between the two is that they use different values for the +`RUSTUP_UPDATE_ROOT` environment variable: + +- A beta release is deployed on `https://dev-static.rust-lang.org/rustup`. +- A stable release is deployed on `https://static.rust-lang.org/rustup`. + +By switching between those two values, Rustup effectively provides two +"self update channels", making beta testing possible with `rustup self update`. + +### Release Artifacts + +The release artifacts are produced by CI when a new commit is merged into the +`master` branch, and they are uploaded to the `rustup-builds` bucket on S3. +There, they are put into a folder named after their commit hash, for example the +artifacts for commit `1cf1b5a` would be uploaded to +`s3://rustup-builds/1cf1b5a`. + +When a new `beta` release is cut, the artifacts are copied to two new locations +in the `dev-static` bucket on S3: + +- One copy is put into an archive named after the version, e.g. + `/rustup/archive/1.0.0`. +- Another copy is put into the `/rustup/dist` folder, which is where clients + look for new versions. + +When a new `stable` release is cut, the artifacts are copied to the `static` +bucket on S3, following the same process as the `beta` release: + +- One copy is archived into `/rustup/archive/1.0.0`. +- Another copy is put into `/rustup/dist`. + +This ensures backwards compatibility with the old release process, while also +ensuring that release artifacts are not overwritten by new builds. + +### Automation + +The interaction with the release artifacts is fully automated. + +First, artifacts are produced automatically by +the [`CI`](https://github.com/rust-lang/rustup/blob/master/.github/workflows/ci.yaml) +job on GitHub Actions when a new commit is merged into the `stable` branch, and +are then automatically uploaded to their respective S3 bucket by the action as +well. + +Second, when making a release, the artifacts are copied to their final locations +by the [promote-release] tool. This reduces the risk of human error and ensures +that the release process is consistent and reliable. The tool is also run in a +secure environment on AWS CodeBuild, reducing the risk of leaking sensitive +credentials that would give write access (past) releases. + +For a `beta` release, `promote-release` performs the following actions: + +1. Query GitHub's API to get the latest commit on the `stable` branch +2. Confirm that `s3://rustup-builds/${commit}` exists +3. Get the new version number from the `stable` branch + 1. Download `Cargo.toml` from `stable` + 2. Parse the file and read the `version` field +4. Confirm that `s3://dev-static/rustup/archive/${version}` does not exist yet +5. Copy `s3://rustup-builds/${commit}` to `s3://dev-static/rustup/archive/${version}` +6. Copy `s3://rustup-builds/${commit}` to `s3://dev-static/rustup/dist` +7. Generate a new manifest and upload it to `s3://dev-static/rustup/dist` + +For a new `stable` release, the process is the same. The only difference is that +the steps 4-6 copy the artifacts to the `static` bucket. + +## How to Cut a Release + +This section documents the steps that a maintainer should follow to cut a new +release of Rustup. + +### 1. Test `rustup-init.sh` + +Before cutting a release, ensure that `rustup-init.sh` is behaving correctly, and that you're satisfied that nothing in the ecosystem is breaking because of the update. A useful set of things to check includes verifying that real-world toolchains install okay, and that `rust-analyzer` isn't broken by the release. While it's not our responsibility if they depend on non-stable APIs, we should behave well if we can. -As a maintainer, you have two options to choose from when cutting a new -release: a beta release or an official release. -The main difference between the two is that they use different values for -the `RUSTUP_UPDATE_ROOT` environment variable: -- A beta release is deployed on `https://dev-static.rust-lang.org/rustup`. -- An official release is deployed on `https://static.rust-lang.org/rustup`. - -By switching between those two values, Rustup effectively provides two "self -update channels", making beta testing possible with `rustup self update`. - -Producing the final release artifacts is a bit involved because of the way -Rustup is distributed. -Below is a list of things to be done in order to cut a new [b]eta release -or an official [r]elease: - -1. [b/r] In a separate PR: - 1. If the version strings in `Cargo.toml`s haven't been updated: - - Decide what the new version number `$VER_NUM` should be. - > **Note:** We always increment the *minor* number unless: - > - A major incompatibility has been introduced in this release: - > increment the *major* number instead. - > - This release is a hotfix because the last one had a defect: - > increment the *patch* number instead. - - Update `Cargo.toml` and `download/Cargo.toml` to have that same new - version number, then run `cargo build` and review `Cargo.lock` changes. +### 2. Update Version + +The release process gets metadata from the `Cargo.toml` file, so ensure that +the version number in `Cargo.toml` is correct. + +1. If the version strings in `Cargo.toml`s haven't been updated: + - Decide what the new version number `$VER_NUM` should be. + > **Note:** We always increment the *minor* number unless: + > - A major incompatibility has been introduced in this release: + > increment the *major* number instead. + > - This release is a hotfix because the last one had a defect: + > increment the *patch* number instead. + - Update `Cargo.toml` and `download/Cargo.toml` to have that same new + version number, then run `cargo build` and review `Cargo.lock` changes. If all looks well, make a commit. - 2. Update `CHANGELOG.md` accordingly if necessary. -2. [b/r] After merging the PR made in step 1, in a separate PR: - 1. Update the commit shasum in `rustup-init.sh` to match the latest commit - on `master`. -3. [b/r] After merging the PR made in step 2, sync `master` to `stable` using - `--ff-only`: - - `git fetch origin master:master` - - `git checkout stable && git merge --ff-only master` - - `git push origin HEAD:stable` -4. [b/r] While you wait for green CI on `stable`, double-check the - functionality of `rustup-init.sh` and `rustup-init` just in case. -5. [b/r] Ensure all of CI is green on the `stable` branch. - Once it is, check through a representative proportion of the builds looking - for the reported version statements to ensure that - we definitely built something cleanly which reports as the right version - number when run `--version`. -6. [r] Make a new PR to the [Rust Blog] adding a new release announcement post. -7. [b/r] Ping someone in the release team to perform the actual release. - They can find instructions in `ci/sync-dist.py`. - > **Note:** Some manual testing occurs here, so hopefully they'll catch - anything egregious in which case abort the change and roll back. -8. [b] Once the beta release has happened, post a new topic named "Seeking beta - testers for Rustup $VER_NUM" on the [Internals Forum]. -9. [r] Once the official release has happened, prepare and push a tag on the - latest `stable` commit. - - `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG - signing the tag) - - `git push origin $VER_NUM` - -[Rust Blog]: https://github.com/rust-lang/blog.rust-lang.org -[Internals Forum]: https://internals.rust-lang.org +2. Update `CHANGELOG.md` accordingly if necessary. + +Submit the changes in a PR and merge it. + +### 3. Update Checksum + +After merging the PR in the previous step, update the commit SHA checksum in +`rustup-init.sh` to match the latest commit on `master`. Submit the change in a +PR and merge it. + +### 4. Sync `master` to `stable` + +After merging the PR in the previous step, sync `master` to `stable` using +`--ff-only`: + +```shell +git fetch origin master:master +git checkout stable && git merge --ff-only master +git push origin HEAD:stable +``` + +This will kick off a new build on GitHub Actions, which will produce the release +artifacts, upload them to `dev-static`, and make the new `beta` release +available at `RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup`. + +### 5. Test the `beta` Release + +While you wait for green CI on `stable`, double-check the functionality of +`rustup-init.sh` and `rustup-init` just in case. + +Ensure all of CI is green on the `stable` branch. Once it is, check through a +representative proportion of the builds looking for the reported version +statements to ensure that we definitely built something cleanly which reports as +the right version number when run `--version`. + +Once the beta release has happened, post a new topic named "Seeking beta testers +for Rustup $VER_NUM" on the [Internals Forum]. + +### 6. Prepare Blog Post for `stable` Release + +Make a new PR to the [Rust Blog] adding a new release announcement post. + +### 7. Request the `stable` Release + +Ping someone in the release team to perform the `stable` release. + +They will have to start a new CodeBuild job on AWS to run the `promote-release` +tool for Rustup. + +### 8. Tag the `stable` Release + +Once the official release has happened, prepare and push a tag on the latest +`stable` commit. + +- `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG + signing the tag) +- `git push origin $VER_NUM` + +[internals forum]: https://internals.rust-lang.org +[promote-release]: https://github.com/rust-lang/promote-release +[rust-lang/rustup]:https://github.com/rust-lang/rustup +[rust blog]: https://github.com/rust-lang/blog.rust-lang.org +