[SQL] Alter Catalog and Table statements

[ahmedabu98]

Addresses #35637

Adds the following features:

Modify a catalog's properties

ALTER CATALOG my_catalog 
[ RESET ('prop1', 'prop2', ...) ] 
[ SET ('prop5'='val5', 'prop6'='val6', ...) ]

Modify a table's properties, schema, or partition spec

ALTER TABLE my_table
[ DROP COLUMNS (col1, col2, ...) ]
[ ADD COLUMNS (col5 VARCHAR COMMENT 'some-comment', col6 INTEGER, ...) ]
[ DROP PARTITIONS ('col1', 'month(col4)', ...) ]
[ ADD PARTITIONS ('truncate(col5, 3)', 'col4', ...) ]
[ RESET ('prop1', 'prop2', ...) ] 
[ SET ('prop5'='val5', 'prop6'='val6', ...) ]

Limitations:

• Does not support adding a required/non-null column to an Iceberg table (introduced in V3). Iceberg requires such a column to specify a default value (so that existing rows can adapt that value to their new column), but Beam Schemas do not support a "default value"

[github-actions]

Assigning reviewers:

R: @Abacn for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[ahmedabu98]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces support for ALTER CATALOG and ALTER TABLE statements in Beam SQL, which is a significant feature addition. The changes are extensive, spanning the SQL parser, DDL execution logic, and the Iceberg IO connector. The implementation is well-structured and accompanied by a comprehensive set of tests for the new functionality.

My review has identified a functional bug in the unparse logic for ALTER TABLE, a potential thread-safety issue in the Iceberg catalog configuration, and an opportunity for code simplification by removing duplicated logic. After addressing these points, this pull request will be in excellent shape.

[github-actions]

Reminder, please take a look at this pr: @Abacn

[Abacn]

assign to next reviewer

[github-actions]

Assigning reviewers:

R: @kennknowles for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

[github-actions]

Reminder, please take a look at this pr: @kennknowles

[github-actions]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[ahmedabu98]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces ALTER CATALOG and ALTER TABLE statements to Beam SQL, enhancing DDL capabilities. The changes include new parser rules, DDL node classes, and interface methods to support these operations. Specific implementations are provided for Iceberg catalogs and tables, as well as a test table provider. The new functionality allows modifying catalog properties, and for tables, it supports updating properties, schema (adding/dropping columns), and partition specifications. Comprehensive unit tests have been added to validate the new features, including a specific test for the limitation of not adding required columns to existing Iceberg tables. The code is well-structured and follows good practices for extensibility.

[github-actions]

Reminder, please take a look at this pr: @Abacn

[github-actions]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions]

Reminder, please take a look at this pr: @chamikaramj

[github-actions]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @Abacn for label java.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions]

Reminder, please take a look at this pr: @Abacn

[github-actions]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions]

Reminder, please take a look at this pr: @kennknowles

[github-actions]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @chamikaramj for label java.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[github-actions]

Reminder, please take a look at this pr: @chamikaramj

[github-actions]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java.

Available commands:

• stop reviewer notifications - opt out of the automated review tooling
• remind me after tests pass - tag the comment author after tests pass
• waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

[Abacn]

Thanks!

[Abacn]

Just a question - these DDLs are added in the main sql module. Are they now supported by Beam SQL in general, or is just supported for Icebeg Catalog and Tables?

[ahmedabu98]

This class just funnels the properties down to the TableProvider implementation.

Currently, only the InMemory provider and Iceberg providers support Alter operations. Other providers will throw an UnsupportedOperationException

[codecov]

Codecov Report

❌ Patch coverage is 77.97834% with 61 lines in your changes missing coverage. Please review.
✅ Project coverage is 36.33%. Comparing base (0274331) to head (9fd7054).
⚠️ Report is 234 commits behind head on master.

Files with missing lines	Patch %	Lines
.../sdk/extensions/sql/impl/parser/SqlAlterTable.java	73.33%	19 Missing and 13 partials ⚠️
...dk/extensions/sql/impl/parser/SqlAlterCatalog.java	74.60%	14 Missing and 2 partials ⚠️
...ions/sql/meta/provider/test/AlterTestTableOps.java	88.09%	3 Missing and 2 partials ⚠️
...am/sdk/extensions/sql/impl/parser/SqlDdlNodes.java	82.60%	0 Missing and 4 partials ⚠️
...dk/extensions/sql/meta/provider/TableProvider.java	0.00%	2 Missing ⚠️
...k/extensions/sql/meta/store/InMemoryMetaStore.java	50.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #36571      +/-   ##
============================================
- Coverage     40.38%   36.33%   -4.05%     
+ Complexity     3476     1690    -1786     
============================================
  Files          1226     1066     -160     
  Lines        188553   167215   -21338     
  Branches       3607     1227    -2380     
============================================
- Hits          76138    60758   -15380     
+ Misses       109012   104259    -4753     
+ Partials       3403     2198    -1205     

Flag	Coverage Δ
java	67.32% <77.97%> (-3.33%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.