Support catalog name in table identifier during load, rename, drop, and purge #150
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related Issue: #123
Validations Run:
make test
make test-integration
make lint
Description:
This PR fixes a bug where
Catalog
APIs returning a table (e.g.catalog.load_table()
,catalog.create_table()
, etc.) return aTable
whose identifier has the catalog name prepended, but providing this identifier to catalog APIs to locate this table raises aNoSuchTable
error due to the prepended catalog name.This implementation simply removes the prepended catalog name from the identifier whenever it is composed of at least 3 elements and the catalog name matches the current catalog implementation in use.
It currently only removes the catalog name in cases where a table identifier refers to the location of an existing table, such as in
catalog.load_table()
, thefrom_identifier
ofcatalog.rename_table()
,catalog.drop_table()
, andcatalog.purge_table()
.One side-effect of this implementation decision is illustrated in the following code, which fails to load a table with a 2-part namespace whose first element is identical to the catalog name:
In this case, we could fallback to again trying to load the table with the catalog name included after catching the
NoSuchTable
error, but this may result in us loading a different table than intended since we don't know if the first identifier element is meant to refer to a namespace or a catalog name.I'm curious to hear reviewers opinions about how concerning these types of side-effects are, potential fixes, and if this calls for further refactoring to add the missing context in the form of identifier metadata, type-differentiated identifier elements, etc.