Add option to grant public read access on Chronon-created tables - #1152
Open
gsarthakdev wants to merge 2 commits into
Open
Add option to grant public read access on Chronon-created tables#1152gsarthakdev wants to merge 2 commits into
gsarthakdev wants to merge 2 commits into
Conversation
Introduces spark.chronon.table_write.grant_public_access (default false) and spark.chronon.table_write.grant_public_access.role (default PUBLIC). When enabled, TableUtils issues a GRANT SELECT right after a table is created via insertPartitions or insertUnPartitioned, so consumers (notebooks, dashboards, downstream jobs) can read Chronon-created tables without a manual grant. A failed grant is logged and does not block table creation, same as the alterTableProperties pattern this mirrors. Closes airbnb#1033
…second one Spark's session builder merges new config into any already-active session in the JVM, so building a second SparkSession with grant_public_access=true in testGrantPublicAccessDoesNotBlockTableCreation was leaking that flag onto the shared session used across the rest of TableUtilsTest, causing testGrantPublicAccessDisabledByDefault to fail when run after it in the same JVM (as CI does). Set/reset the conf on the shared session instead.
Author
|
@pengyu-hou Thank you again for scoping this out on #1033. I've implemented the grantPublicAccess approach you suggested there (mirroring alterTableProperties), and CI is passing. Marking this ready for review, let me know if you'd like anything adjusted. |
gsarthakdev
marked this pull request as ready for review
September 11, 2026 02:50
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Chronon-created tables default to permissions restricted to the creating service account, so other consumers (notebooks, dashboards, downstream jobs) can't read them without a manual grant.
Add a
grantPublicAccessconfig option toTableUtils, invoked right after table creation, that issues aGRANT SELECTto a configurable role.Summary
TableUtilscreates tables in two places,insertPartitionsandinsertUnPartitioned, viacreateTableSql+sql(...). Neither path grants any access beyond the creator.This PR adds:
spark.chronon.table_write.grant_public_access(defaultfalse) - enables the grantspark.chronon.table_write.grant_public_access.role(defaultPUBLIC) - the role to grantSELECTtoTableUtils.grantPublicAccess(tableName), called right after table creation in bothinsertPartitionsandinsertUnPartitionedwhen the flag is enabledA failed grant is caught and logged as a warning rather than propagated, so it can't block table creation or writes - same non-critical-failure pattern as
alterTableProperties.Why / Goal
Filed and scoped in #1033. @pengyu-hou confirmed the use case and suggested this approach (
grantPublicAccess, modeled onalterTableProperties) there; this PR implements it as discussed.Test Plan
TableUtilsTest.testGrantPublicAccessDisabledByDefault,TableUtilsTest.testGrantPublicAccessDoesNotBlockTableCreation)Closes #1033