[airbyte-ci/auto-merge] - Updates PR pipeline to bypass CI checks for promoted release candidate PRs #49827

pnilan · 2024-12-17T00:23:37Z

What

Adds new auto-merge/bypass-checks label to publishing pipeline when publishing with --promote-release-candidate flag.
auto-merge workflow includes label mapping to required validator methods. If new autom-merge/byapss-checks label included in a given PR, auto-merge will run all validator methods except for the CI checks one.
Closes https://github.com/airbytehq/airbyte-internal-issues/issues/11168

How

New auto-merge/bypass-ci-checks label added to PR when running publish pipeline with --promote-release-candidate flag. Also new promoted-rc label added for human filtering of PRs
Checks the PR against a mapped set of PR validators based on included labels within the PR.
- New get_pr_validators method checks the PR's labels against the new VALIDATOR_MAPPING and returns mapped pr validator methods.

Review guide

publish/pipeline.py
auto_merge/main.py
auto_merge/pr_validators.py
auto_merge/consts.py

Can this PR be safely reverted and rolled back?

YES 💚
NO ❌

vercel · 2024-12-17T00:23:41Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jan 9, 2025 10:44pm

clnoll

@pnilan this looks good to me, just a couple of very minor comments. I'll leave the final approval to @alafanechere.

clnoll · 2024-12-18T23:24:44Z

airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/pipeline.py

@@ -753,7 +753,7 @@ async def run_connector_promote_pipeline(context: PublishConnectorContext, semap
                    all_modified_files.update(
                        await add_changelog_entry.export_modified_files(context.connector.local_connector_documentation_directory)
                    )
-                post_changelog_pr_update = CreateOrUpdatePullRequest(context, skip_ci=False, labels=["auto-merge"])
+                post_changelog_pr_update = CreateOrUpdatePullRequest(context, skip_ci=True, labels=["auto-merge", "auto-merge/promoted-rc"])


Can "auto-merge" and "auto-merge/promoted-rc" be pulled in from the constants file?

We could ideally add the auto_merge packages as a dependency of pipelines and import constant from it.

Updated to add automerge as a dependency.

clnoll · 2024-12-18T23:27:45Z

airbyte-ci/connectors/auto_merge/src/auto_merge/pr_validators.py

@@ -48,12 +48,13 @@ def head_commit_passes_all_required_checks(
    return True, None


-# A PR is considered auto-mergeable if:
+# A standard PR is considered auto-mergeable if:


nit: since we don't define what "standard" means here, I'd prefer to leave this comment as-is.

alafanechere

Thank you @pnilan . The implementation looks correct. Please consider my suggestion which I think would help in the maintenance of the package in case other use cases like this arise in the future. And can you please bump the auto_merge package version in pyproject.toml?

alafanechere · 2024-12-19T15:15:24Z

airbyte-ci/connectors/auto_merge/src/auto_merge/pr_validators.py

+# Let's declare faster checks first as the check_if_pr_is_auto_mergeable function fails fast.
+DEFAULT_ENABLED_VALIDATORS = [has_auto_merge_label, targets_main_branch, only_modifies_connectors, head_commit_passes_all_required_checks]
+PROMOTED_RC_PR_VALIDATORS = [has_auto_merge_label, targets_main_branch, only_modifies_connectors]


nit: I'd reverse the logic.
The default validators would have less requirements than a second one which would require the head commit to pass.

DEFAULT_ENABLED_VALIDATORS = {has_auto_merge_label, targets_main_branch, only_modifies_connectors} PASSING_CHECKS_VALIDATORS = DEFAULT_ENABLED_VALIDATORS + { "head_commit_passes_all_required_checks"}

It would define an implicit logic of strictness elevation.

I agree -- updated.

alafanechere · 2024-12-19T15:18:43Z

airbyte-ci/connectors/auto_merge/src/auto_merge/consts.py

 AUTO_MERGE_LABEL = "auto-merge"
+AUTO_MERGE_PROMOTED_RC_LABEL = "auto-merge/promoted-rc"


I suggest removing the auto-merge/ prefix and defining a mapping of labels to validator groups. This would allow to easily expand your implementation to other use cases.
E.g : A PR has the auto-merge label AND promoted-rc -> use a specific set of validators. We could then add different combination with only a single change to make to a mapping.

I've updated this by adding a get_pr_validators function and a label -> validator mapping dictionary. Take a look and let me know what you think.

alafanechere · 2024-12-19T15:19:45Z

airbyte-ci/connectors/pipelines/pipelines/airbyte_ci/connectors/publish/pipeline.py

@@ -753,7 +753,7 @@ async def run_connector_promote_pipeline(context: PublishConnectorContext, semap
                    all_modified_files.update(
                        await add_changelog_entry.export_modified_files(context.connector.local_connector_documentation_directory)
                    )
-                post_changelog_pr_update = CreateOrUpdatePullRequest(context, skip_ci=False, labels=["auto-merge"])
+                post_changelog_pr_update = CreateOrUpdatePullRequest(context, skip_ci=True, labels=["auto-merge", "auto-merge/promoted-rc"])


We could ideally add the auto_merge packages as a dependency of pipelines and import constant from it.

pnilan · 2025-01-03T17:08:25Z

airbyte-ci/connectors/pipelines/pipelines/helpers/utils.py

@@ -115,7 +115,7 @@ async def get_file_contents(container: Container, path: str) -> Optional[str]:
 def catch_exec_error_group() -> Generator:
    try:
        yield
-    except anyio.ExceptionGroup as eg:
+    except ExceptionGroup as eg:


With the inclusion of auto-merge as a library, I needed to update the anyio version to at least 4.0.0, which deprecated anyio.ExceptionGroup in favor of the built-in ExceptionGroup. See here: https://anyio.readthedocs.io/en/stable/migration.html#task-groups-now-wrap-single-exceptions-in-groups

alafanechere

LGTM!

alafanechere · 2025-01-09T20:47:28Z

airbyte-ci/connectors/auto_merge/src/auto_merge/pr_validators.py

+# Let's declare faster checks first as the check_if_pr_is_auto_mergeable function fails fast.
+VALIDATOR_MAPPING: dict[str, list[Callable]] = {
+    AUTO_MERGE_LABEL: [has_auto_merge_label, targets_main_branch, only_modifies_connectors, head_commit_passes_all_required_checks],
+    AUTO_MERGE_BYPASS_CI_CHECKS_LABEL: [has_auto_merge_bypass_ci_checks_label, targets_main_branch, only_modifies_connectors],
+}


nit for readability

Suggested change

# Let's declare faster checks first as the check_if_pr_is_auto_mergeable function fails fast.

VALIDATOR_MAPPING: dict[str, list[Callable]] = {

AUTO_MERGE_LABEL: [has_auto_merge_label, targets_main_branch, only_modifies_connectors, head_commit_passes_all_required_checks],

AUTO_MERGE_BYPASS_CI_CHECKS_LABEL: [has_auto_merge_bypass_ci_checks_label, targets_main_branch, only_modifies_connectors],

}

COMMON_VALIDATORS = {

has_auto_merge_label,

targets_main_branch,

only_modifies_connectors,

}

# Let's declare faster checks first as the check_if_pr_is_auto_mergeable function fails fast.

VALIDATOR_MAPPING: dict[str, set[Callable]] = {

AUTO_MERGE_LABEL: COMMON_VALIDATORS + {head_commit_passes_all_required_checks},

AUTO_MERGE_BYPASS_CI_CHECKS_LABEL: COMMON_VALIDATORS,

}

pnilan added 2 commits December 16, 2024 16:21

updates pipeline to bypass ci checks for release candidate promotion PRs

8e6e620

bumps patch version

bc5bf9c

Merge branch 'master' into pnilan/workflows/update-automerge-rules

6f8375b

vercel bot deployed to Preview December 17, 2024 00:34 View deployment

Merge branch 'master' into pnilan/workflows/update-automerge-rules

de0340a

vercel bot deployed to Preview December 17, 2024 22:25 View deployment

add test case fail

d87dd8c

vercel bot deployed to Preview December 17, 2024 22:45 View deployment

add temp logging

aaeb40c

vercel bot deployed to Preview December 18, 2024 22:25 View deployment

update conditional

e8b0af5

vercel bot deployed to Preview December 18, 2024 22:47 View deployment

pnilan marked this pull request as ready for review December 18, 2024 23:08

pnilan requested a review from a team as a code owner December 18, 2024 23:08

vercel bot deployed to Preview December 18, 2024 23:13 View deployment

clnoll reviewed Dec 18, 2024

View reviewed changes

alafanechere approved these changes Dec 19, 2024

View reviewed changes

vercel bot deployed to Preview December 20, 2024 18:29 View deployment

vercel bot deployed to Preview December 20, 2024 18:48 View deployment

pnilan requested a review from alafanechere December 20, 2024 19:09

vercel bot deployed to Preview December 20, 2024 19:29 View deployment

pnilan force-pushed the pnilan/workflows/update-automerge-rules branch from c4e0dd9 to 7b0d997 Compare December 20, 2024 19:42

vercel bot deployed to Preview December 20, 2024 19:51 View deployment

vercel bot deployed to Preview December 20, 2024 21:24 View deployment

vercel bot deployed to Preview December 30, 2024 18:22 View deployment

vercel bot deployed to Preview January 2, 2025 22:13 View deployment

vercel bot deployed to Preview January 2, 2025 23:40 View deployment

vercel bot deployed to Preview January 3, 2025 00:14 View deployment

pnilan commented Jan 3, 2025

View reviewed changes

pnilan added 5 commits January 8, 2025 17:06

update for anyio migration

ef46b14

revert comment

2047076

add bypass-specific label for promomted release candidates

de2a15f

chore: format code

3d04e34

Merge branch 'master' into pnilan/workflows/update-automerge-rules

62cf3e7

vercel bot deployed to Preview January 9, 2025 01:21 View deployment

pnilan added 2 commits January 9, 2025 08:56

Merge branch 'master' into pnilan/workflows/update-automerge-rules

c1f3030

fix typo in pyproject

a540586

vercel bot deployed to Preview January 9, 2025 17:17 View deployment

Merge branch 'master' into pnilan/workflows/update-automerge-rules

f93a5ca

vercel bot deployed to Preview January 9, 2025 17:57 View deployment

pnilan added 2 commits January 9, 2025 10:11

update poetry lock

71ba6ee

remove temporary log

ddbd5c0

vercel bot deployed to Preview January 9, 2025 18:24 View deployment

pnilan added 2 commits January 9, 2025 10:36

chore: type-check/format

56133b6

chore: format

54bdd0b

pnilan added the area/airbyte-ci label Jan 9, 2025

vercel bot deployed to Preview January 9, 2025 18:49 View deployment

set develop to true for auto-merge

f31d9ca

pnilan requested a review from alafanechere January 9, 2025 19:13

vercel bot deployed to Preview January 9, 2025 19:19 View deployment

alafanechere approved these changes Jan 9, 2025

View reviewed changes

pnilan added 2 commits January 9, 2025 13:09

chore: readability

9299894

chore: format code

7213b48

vercel bot deployed to Preview January 9, 2025 21:16 View deployment

vercel bot deployed to Preview January 9, 2025 21:41 View deployment

chore: type_check and bump patch version

915aad5

vercel bot deployed to Preview January 9, 2025 22:44 View deployment

pnilan merged commit 4265fab into master Jan 9, 2025
29 checks passed

pnilan deleted the pnilan/workflows/update-automerge-rules branch January 9, 2025 23:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[airbyte-ci/auto-merge] - Updates PR pipeline to bypass CI checks for promoted release candidate PRs #49827

[airbyte-ci/auto-merge] - Updates PR pipeline to bypass CI checks for promoted release candidate PRs #49827

pnilan commented Dec 17, 2024 •

edited

Loading

vercel bot commented Dec 17, 2024 •

edited

Loading

clnoll left a comment

clnoll Dec 18, 2024

alafanechere Dec 19, 2024

pnilan Dec 20, 2024

clnoll Dec 18, 2024

pnilan Dec 20, 2024

alafanechere left a comment

alafanechere Dec 19, 2024

pnilan Dec 20, 2024

alafanechere Dec 19, 2024 •

edited

Loading

pnilan Dec 20, 2024

alafanechere Dec 19, 2024

pnilan Jan 3, 2025

alafanechere left a comment

alafanechere Jan 9, 2025

		AUTO_MERGE_LABEL = "auto-merge"
		AUTO_MERGE_PROMOTED_RC_LABEL = "auto-merge/promoted-rc"

[airbyte-ci/auto-merge] - Updates PR pipeline to bypass CI checks for promoted release candidate PRs #49827

[airbyte-ci/auto-merge] - Updates PR pipeline to bypass CI checks for promoted release candidate PRs #49827

Conversation

pnilan commented Dec 17, 2024 • edited Loading

What

How

Review guide

Can this PR be safely reverted and rolled back?

vercel bot commented Dec 17, 2024 • edited Loading

clnoll left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alafanechere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alafanechere Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alafanechere left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pnilan commented Dec 17, 2024 •

edited

Loading

vercel bot commented Dec 17, 2024 •

edited

Loading

alafanechere Dec 19, 2024 •

edited

Loading