Add a beta-promotion picker to release builds to the Play Store beta track#23079
Add a beta-promotion picker to release builds to the Play Store beta track#23079oguzkocer wants to merge 14 commits into
Conversation
Generated by 🚫 Danger |
|
|
|
|
First step of the beta-promotion picker: read the beta track's current version codes for WordPress and Jetpack via google_play_track_version_codes and log them plus the combined floor. Read-only for now.
Extend gather_beta_candidates to read every uploaded AAB version code per app via the Play app bundle explorer (Supply::Client#aab_version_codes over a throwaway edit), then compute the codes present for both apps and above the beta floor. Read-only; logs the floor, the pool, and the candidates.
gather_beta_candidates now opens a Buildkite block step (candidate list posted to Slack) and promote_to_beta promotes the chosen version code to the beta track for WordPress and Jetpack as a draft, referencing the existing bundle (no rebuild). Adds the Buildkite command scripts and the promote-beta.yml pipeline.
The beta-track and bundle-explorer lookups rescued every error into [], which is indistinguishable from a legitimately empty result: a transient failure would collapse the candidate intersection (reading as 'nothing to promote') or null out the floor (reading as 'promote everything'). Raise a clear error instead, mirroring the iOS promote lanes. Also wrap the throwaway edit's read in ensure so it's aborted even when the read raises.
upload_promotion_steps hardcoded build/promote-steps.yml while PROMOTION_STEPS_FILE built the same path from PROJECT_ROOT_FOLDER; the two could silently diverge. Both now derive from PROMOTION_STEPS_RELATIVE_PATH.
buildkite_api_get had no open/read timeout, so a hung response could stall the gather lane across all 5 block-step poll attempts. Bound both to 10s.
The promote path hand-rolled its own upload_to_play_store call, duplicating the retry, skip flags, and retained pinned codes that upload_build_to_play_store already owns. Generalize that lane to also promote an existing build by version_code (reference the code, skip the AAB), and have the beta promotion call it. The pinned RETAINED_VERSION_CODES constant now lives in one place.
Use Array#max(n) instead of sort.reverse.first(n) for the newest N candidates, and reject a non-integer version_code up front with a clear message rather than letting Integer() fail per app.
Route the candidate list, per-app results, and the pipeline failure-notify to #test-wpmobile-slack-integration while validating the flow. Revert to #build-and-ship before merge.
Cut over-explaining and session-flavored comments down to the factual point, and drop a stale comment that claimed available_aab_version_codes returns [] on failure (it raises).
- Candidates are the version code itself; drop the version_name/build_number decode, which was wrong for hotfix (patch) versions and unused beyond the block-step/Slack label. - Move the generic send_slack_message/notify_slack/get_required_env helpers out of promote.rb into the Fastfile. - Keep [1440] inline in upload_build_to_play_store and drop the extracted RETAINED_VERSION_CODES constant (out of scope now that promote reuses the lane). - Raise the Buildkite API poll timeout from 10s to 30s.
Replace the ad-hoc buildkite_api_token helper and the lone SLACK_WEBHOOK read with the shared get_required_env helper (matching wpios), so required secrets are read one consistent way. Optional env vars keep ENV.fetch with a default.
upload_to_play_store(version_code:, skip_upload_aab: true) commits an empty edit — supply only builds a track release from binaries uploaded in the same run, so the bare version_code was ignored and no beta release was created. Create the draft beta release directly with Supply::Client (build a TrackRelease for the chosen version code, set it on the beta track, commit), mirroring supply's own update_track. Revert upload_build_to_play_store to its original AAB-only form.
…script shebangs - Replace inline [1440] in upload_build_to_play_store and promote_version_code_to_beta with PLAY_STORE_VERSION_CODES_TO_RETAIN - Change gather-beta-candidates.sh and promote-to-beta.sh shebangs to '#!/usr/bin/env bash' with 'set -eu' - Update promote-beta.yml notify and send_slack_message channel comments
oguzkocer
force-pushed
the
task/promote-to-beta
branch
from
ce4b529 to
83e0757
Compare
AliSoftware
left a comment
AliSoftware
left a comment
There was a problem hiding this comment.
I haven't tested the process yet (only read the PR diff), but this seems like an interesting idea! Looks good in principle 🙂
One question: I'm curious how the promote-beta.yml pipeline is planned to be triggered in practice? Do you plan to adjust the ReleasesV2 scenario to add a dedicated task at some specific points in the release process to trigger this?
(sorry if I missed it; I couldn't find the info in the PR description nor an associated Linear issue that would give more details)
| def available_aab_version_codes(package_name:) | ||
| require 'supply' | ||
| require 'supply/options' | ||
|
|
||
| Supply.config = FastlaneCore::Configuration.create( | ||
| Supply::Options.available_options, | ||
| { json_key: UPLOAD_TO_PLAY_STORE_JSON_KEY, package_name: package_name } | ||
| ) | ||
|
|
||
| client = Supply::Client.make_from_config | ||
| client.begin_edit(package_name: package_name) | ||
| codes = | ||
| begin | ||
| client.aab_version_codes | ||
| ensure | ||
| # Always discard the throwaway edit, even if the read raises. | ||
| client.abort_current_edit | ||
| end | ||
| Array(codes).compact.map(&:to_i) | ||
| rescue StandardError => e | ||
| # Raise rather than return [], as in beta_track_version_codes. | ||
| UI.user_error!("Unable to list the available AAB version codes for #{package_name}: #{e.message}") | ||
| end |
There was a problem hiding this comment.
Nice!
💡 This seems like a good candidate to DRY/wrap this into a proper fastlane action in release-toolkit so that other Android repos could also adopt it (especially if this approach of promoting beta works well in WPAndroid and the idea might gain traction in other Android repos 🙂 )
Of course, not a blocker for this PR, just a note for cleaning things up nicely later, once this starts to be proven working well.
| def promote_version_code_to_beta(package_name:, version_code:) | ||
| require 'supply' | ||
| require 'supply/options' | ||
|
|
||
| Supply.config = FastlaneCore::Configuration.create( | ||
| Supply::Options.available_options, | ||
| { json_key: UPLOAD_TO_PLAY_STORE_JSON_KEY, package_name: package_name, track: BETA_TRACK } | ||
| ) | ||
|
|
||
| client = Supply::Client.make_from_config | ||
| client.begin_edit(package_name: package_name) | ||
|
|
||
| committed = false | ||
| begin | ||
| release = AndroidPublisher::TrackRelease.new( | ||
| # TODO: switch to 'completed' once the feature is ready to distribute to beta testers. | ||
| status: 'draft', | ||
| # Keep the pinned legacy code(s) on the track, same as the AAB-upload path. | ||
| version_codes: [Integer(version_code), *PLAY_STORE_VERSION_CODES_TO_RETAIN] | ||
| ) | ||
| track = client.tracks(BETA_TRACK).first || AndroidPublisher::Track.new(track: BETA_TRACK) | ||
| track.releases = [release] | ||
|
|
||
| client.update_track(BETA_TRACK, track) | ||
| client.commit_current_edit! | ||
| committed = true | ||
| ensure | ||
| # Discard the edit if we bailed before committing (a committed edit can't be aborted). | ||
| client.abort_current_edit unless committed | ||
| end | ||
| end |
There was a problem hiding this comment.
Ditto, not a blocker but thinking this is great candidate to make as a dedicated fastlane action in release-toolkit in the future 💡
| # The default is intentionally the test channel while the continuous-release process is being built | ||
| # out; it moves to #build-and-ship once the flow ships to real testers. | ||
| def send_slack_message(message:, channel: '#test-wpmobile-slack-integration') |
There was a problem hiding this comment.
Leaving a 🎗️ reminder comment just to surface this more visibly so that we don't forget updating this temp value once we have properly tested the PR before merging.
There was a problem hiding this comment.
I think we actually want to leave it like this for now. Not all of the release from trunk model have been implemented yet and until we are ready to go to "production" with it, I'd rather keep it on as the test channel and freely test out the various other steps without worrying about the noise. So, I'd like to treat this as a feature flag for now.
We also can't really forget to fix this for any meaningful amount of time since we'll notice it going to the other channel and there is really no harm if the first or second time in production it goes to the test channel as it'll still work.
@AliSoftware It'll be on a weekly schedule I believe. |
Oh, I see, interesting! 👍 The main feedback and question I'd raise about this approach then, is just one about paper-trail. ReleasesV2 is supposed to be the tool acting as the source of truth for recording what was released and when for which version; so I wonder if we could find a way for this approach to report back to the ReleasesV2 database which build has been promoted and when 🤔 IMHO this paper trail in Rv2 is important not only for release managers but also so that product leads and others in the company can follow-along what what released and when, so that if a question arise about "what is the latest release at the moment ?" / "what's the state of release version x.y, is it out to end users already?" / "when was build XYZ released to end users?", or even just get a timeline when writing a post-mortem, we have a record in ReleasesV2 too — it being the official source-of truth for everyone about releases data — and can thus track it back to a date, a CI job that did the compilation of that build, and a commit. 🤔 💭 💡 A couple of ideas for approaching this without disrupting the main "promote" approach you're introducing here
We could then imagine e.g.
To be clear, this is just a brainstorming around the idea as I'm potentially late to the party (I was on s8l so sorry if I missed previous discussions that might have already happened about this 😅 ), and probably isn't a blocker for this PR and more something to keep in mind as follow-up for the longer run. |
|
@AliSoftware – we can annotate these Buildkite builds easily enough. If y'all want to come up with a format for this it should be pretty easy to add on whenever – for now though, I'm not sure it's worth blocking this. We still know when we're promoting things live, and if someone needs an answer Claude should be able to provide it pretty easily between Buildkite's MCP and the repo history. |
5 participants
Description
Adds a way to promote an already-uploaded build to the Play Store
betatrack without rebuilding. A developer picks a build from a Buildkite block step, and WordPress and Jetpack are promoted together (they share a version code, so one pick covers both).On the term "promote": this isn't entirely Play Console's "Promote release" (which advances a track's current release). Here we pick any uploaded build and create a
betarelease for it, but it's still promotion up the track ladder. It's the terminology this PR proposes, but happy to take it to another direction if reviewers have a strong preference to use another term.What's included:
fastlane/lanes/promote.rb— two lanes:gather_beta_candidateslists the promotable builds (version codes present in both apps' Play libraries and above the currentbetarelease, newest first), writes and uploads a Buildkite block step to choose one, and posts the candidate list to Slack.promote_to_beta version_code:<code>creates a draftbetarelease for that version code in each app directly via the Play API (Supply::Client): open an edit, set the release on thebetatrack, commit..buildkite/—promote-beta.ymlpluscommands/gather-beta-candidates.shandcommands/promote-to-beta.sh, wiring the gather → block → promote flow. Candidate discovery reads the beta track (google_play_track_version_codes) and the uploaded-bundle list (Supply::Client#aab_version_codesover a throwaway, aborted edit).fastlane/Fastfile— adds thesend_slack_message,notify_slack, andget_required_envhelpers and thePLAY_STORE_VERSION_CODES_TO_RETAINconstant, and imports the new lane file.fastlane/lanes/build.rb—version_codes_to_retainnow references the sharedPLAY_STORE_VERSION_CODES_TO_RETAINconstant.Notes:
release_status: 'draft', so nothing reaches beta testers yet. Flip to'completed'inpromote.rbwhen ready to distribute.trunk(ensure_promotion_on_trunk!); a local or off-branch run fails by design.Testing instructions
Promotion is a Play Store track operation, so there is no app build to install. The flow runs entirely on Buildkite and ends in a draft release in the Play Console (which does not reach testers and can be discarded).
Because the promote lanes refuse to run off
trunk, e2e-test from a throwaway branch with the guard removed:Set up a test branch:
task/promote-to-beta(e.g.task/promote-to-beta-e2e).ensure_promotion_on_trunk!calls infastlane/lanes/promote.rband commit.trunk.Run the picker:
PIPELINE=promote-beta.yml.promote_to_beta.Confirm in the Play Console: