Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] dbt should not call create_schema outside of model materializations #11345

Open
2 tasks done
sdebruyn opened this issue Feb 28, 2025 · 0 comments
Open
2 tasks done
Labels
bug Something isn't working triage

Comments

@sdebruyn
Copy link
Contributor

sdebruyn commented Feb 28, 2025

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

As a user, when I use pre-hook in dbt, I expect this to run before dbt starts creating things in my database. That is - you guessed it - not the case.

Before the pre-hooks are executed, dbt already calls the create_schema(...) macro to create the schema set in the profiles.yml file. You also don't have any model information in the create_schema(...) macro that you could use to override this behaviour. The macro accepts a relation as an argument, but it's only given relations where only the schema is set.

Expected Behavior

dbt should not call create_schema(...) in the beginning of the run. It should call create_schema(...) right before the model materialization happens and then pass the relation it's going to materialize as an argument to this macro. Schemas can be different for every model so creating the initial one listed in profiles.yml might not even be necessary.

Why this matters:

In Trino / Starburst permissions work in a particular way. Users are assigned roles and permissions are assigned to those roles. In most database engines, you would then have the sum of all of your permissions available to you, but not in Starburst. You have to explicitly assume a role you're given before you can use the linked permissions. So in Starburst if I work with schemas which belong to a different role than my default role, I have to first run a pre-hook with SET ROLE rolename.

Steps To Reproduce

  1. Add a pre-hook to a model
  2. dbt run -s that_model
  3. Check dbt logs
  4. Notice that dbt first creates the schema and only then runs the pre-hook

dbt tasks have a [before_run](https://github.com/dbt-labs/dbt-core/blob/f7c4c3c9cc2db8327c3fbb9e4889f7f0f1e9e167/core/dbt/task/runnable.py#L528) function which - as the name implies - runs before anything else. The dbt run has [create_schemas](https://github.com/dbt-labs/dbt-core/blob/f7c4c3c9cc2db8327c3fbb9e4889f7f0f1e9e167/core/dbt/task/run.py#L1006) as part of the implementation of this function.
Can we just remove this call there?

Relevant log output

Environment

dbt-core 1.9

Which database adapter are you using with dbt?

other (mention it in "Additional Context")

Additional Context

dbt-trino, but IMHO irrelevant and should be fixed in dbt-core

@sdebruyn sdebruyn added bug Something isn't working triage labels Feb 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

1 participant