Fix insert_by_period for Postgres #36
Conversation
The insert_by_period materialization depends on dbt_utils getdate() functions. So to get insert_by_period work, dbt_utils must be added to the packages.
The getdate() function is not valid across different warehouses. Therefore, we need to use the corresponding dbt_utils function
Postgres needs subqueries to have an alias
In the creation of the relation node, Postgres cannot work with a schema. Therefore I created a small macro, where I use dispatch to differentiate between postgres and the other warehouses.
|
The fixes for Postgres have now been implemented and tested against a postgres instance. However, I had no chance to test against any other backends. Reviews are very welcome. |
| @@ -12,7 +12,7 @@ | |||
| {{ dateadd('millisecond', | |||
| -1, | |||
| "nullif('" ~ stop_date ~ "','')::timestamp") }}, | |||
| getdate() | |||
| {{ dbt_utils.current_timestamp() }} | |||
There was a problem hiding this comment.
In dbt v1.3, this changed to dbt.current_timestamp() (note dbt not dbt_utils), and in dbt utils 1.0 (coming very very soon) current_timestamp() is being removed. I recommend not creating a dependency on utils and instead using the one provided in the dbt namespace directly.
Note that this would create a requirement of dbt Core 1.3 for this project, which you can enforce by adding a require-dbt-version block https://docs.getdbt.com/reference/project-configs/require-dbt-version
There was a problem hiding this comment.
Thanks for pointing me to this. This is resolved now.
This is needed to resolve dbt.current_timestamp() which was introduced in 1.3.0
|
Thanks @LMandtler! |
The insert_by_period materialization does not work for a postgres target.
Earlier discussions on this topic:
This is fixed in this PR by taking the ideas of the first link above.
Disclaimer: These are not my ideas. I am just a regular user of this materialization and am happy to have it fixed.