Draft
Conversation
…l_workflows() 8 mzcompose.py files had an identical ~8-line workflow_default body that iterates all workflows as test parts. Extract this into a single method on Composition and replace the boilerplate with a one-liner call. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ws() Extend run_all_workflows() to support skipping additional workflows (beyond 'default') and optionally sharding via Buildkite parallelism. This deduplicates workflow_default boilerplate in 4 more test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ters Add internally_sharded, args, and between_workflows parameters to Composition.run_all_workflows() and use them to replace the repeated "internal sharding + buildkite sharding" boilerplate in 5 CDC test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…files() Add `shard_test_files()` and `glob_test_files()` methods to Composition that encapsulate the repeated pattern of parsing a filter argument, globbing for .td files in the test directory, sharding via Buildkite parallelism, and printing the result. Replace 8 occurrences of this ~10-15 line boilerplate across 6 CDC test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…run_scenarios() Deduplicate the repeated buildkite shard + scenario loop pattern across 5 CDC resumption test files (10 occurrences) into a single reusable method on Composition. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move duplicated helper functions (create_mysql, create_mysql_replica, get_targeted_mysql_version, PostgresRecvlogical, create_postgres, get_targeted_pg_version) from test/mysql-cdc and test/pg-cdc pairs into shared utility modules, following the existing mysql_util.py pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…orkflows() Replace the 29-line workflow_default boilerplate in both mysql-cdc-resumption and mysql-cdc-resumption-old-syntax with calls to c.run_all_workflows(), using the exclude, internally_sharded, and between_workflows parameters added in earlier sessions. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…estdrive_size_args() Replace repeated two-line f-string pairs for default replica/storage size testdrive vars with a single static method call, improving readability across 6 CDC and kafka-matrix test files. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ows() Replace manual workflow_default boilerplate in 3 CDC resumption test files with c.run_all_workflows(between_workflows=...). Also extract a run_testdrive_files helper in sql-server-resumption-old-syntax to eliminate 13 repeated SQL Server credential var pairs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…l_cdc_resumption.py The two mysql-cdc-resumption mzcompose.py files (675 lines each) differed by only 1 import line. Extract all 24 shared helper and scenario functions into a new misc/python/materialize/mysql_cdc_resumption.py module, reducing each file from 675 to 313 lines (-324 lines net). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…resumption.py Extract 14 shared functions (helpers + scenarios) from pg-cdc-resumption and pg-cdc-resumption-old-syntax into a new shared module, eliminating ~280 lines of duplicated code. The new-syntax file retains its 3 extra scenarios and pg8000 helpers; the old-syntax file is reduced from 281 to 95 lines. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The SSH setup boilerplate (run setup.td, query public key, authorize on bastion host) was repeated in 10 workflows. Extract three helpers: _get_ssh_public_key, _authorize_ssh_key, _setup_and_authorize_ssh. Net -139 lines (42 added, 181 removed). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add `MySql.default_testdrive_args()` and `SqlServer.default_testdrive_args()` static methods that return the standard testdrive --var args for MySQL root password and SQL Server credentials. Replace all 24 inline occurrences across 19 files with calls to these methods, consistent with the existing `Materialized.default_testdrive_size_args()` pattern. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…s_util.py Move 5 identical replication slot helper functions from both pg-cdc/mzcompose.py and pg-cdc-old-syntax/mzcompose.py into the shared postgres_util.py module. The helpers use the common cursor/execute/fetchall DB interface and work with both psycopg and pg8000 connection objects. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move 4 identical functions (workflow_cdc, workflow_replica_connection, workflow_schema_change_restart, _make_inserts) from both mysql-cdc and mysql-cdc-old-syntax mzcompose.py files into a shared mysql_cdc.py module. The mzcompose framework discovers imported workflow_* functions via getmembers(module, isfunction), so no wrapper functions are needed. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…rive_ssl_args to postgres_util.py Move workflow_wal_level, workflow_replication_disabled, and kill_pg_and_mz into a new shared pg_cdc.py module. Move get_testdrive_ssl_args() from pg-cdc-old-syntax's local definition to postgres_util.py and use it in both pg-cdc files, replacing inline SSL cert extraction in pg-cdc. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… in checks Add a pre-dedented `KAFKA_SCHEMA` constant and a shared `remove_target_cluster_from_explain()` function to `checks/common.py`, replacing 12 identical `schemas()`/`schema()` wrapper functions and 5 identical `remove_target_cluster_from_explain()` definitions across 20 check files. Also replaces 3 inline `dedent(KAFKA_SCHEMA_WITH_SINGLE_STRING_FIELD)` usages with the constant. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…nt_connection_drop into pg_cdc.py Move 3 shared workflow functions from pg-cdc and pg-cdc-old-syntax mzcompose.py files into the shared pg_cdc.py module. Each function is parameterized to handle the minor differences between the two test suites (exclude list, max_replication_slots, DB connection library). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace 27 occurrences of the 4-line ALTER SYSTEM SET boilerplate across 9 test files with a single method call on Composition. This makes each call site clearly express intent rather than exposing port/user details. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…boilerplate
Replace 48 occurrences of the 4-line `c.sql("ALTER SYSTEM SET ...",
port=6877, user="mz_system")` pattern across 16 files with a 1-line
`c.alter_system_set(name, value)` call. The method handles the internal
port and system user automatically and forwards extra kwargs like
`print_statement` and `service`.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pre-merge checklist
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.