[BUG] Syntax for derived columns in stage macro changed

[OGrohmann]

Describe the bug

We are currently using datavault4dbt version 1.4.2 and want to upgrade to the latest version. Unfortunately the build of our stage models is failing with the latest version as the definition of our derived columns is not supported anymore.

Please adapt the macro in a way that the old definition of derived columns is still working, thanks!

Environment

• dbt version: cloud version "latest"
• datavault4dbt version: latest
• Database/Platform: Snowflake

To Reproduce

Example:

We are using following syntax in the derived columns section of the stage models to define constants and hard rules:

derived_columns:
SOURCE_SYSTEM: "!MY_SYSTEM"
DOCUMENT_ID: "LTRIM(DOKNR,0)"

This syntax does not work anymore and has to be changed to following:

derived_columns:
SOURCE_SYSTEM:
VALUE: "!MY_SYSTEM"
DATATYPE: "VARCHAR"
DOCUMENT_ID:
VALUE: "LTRIM(DOKNR,0)"
DATATYPE: "VARCHAR"

Expected behavior

Example:

The following stage syntax for derived columns should successfully be compiled and executed.

derived_columns:
SOURCE_SYSTEM: "!MY_SYSTEM"
DOCUMENT_ID: "LTRIM(DOKNR,0)"

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

[tkirschke]

Hi @OGrohmann and thanks for reaching out!

Sorry to hear that the new version breaks your existing syntax for derived columns. However, the syntax that you use right now was never intended by the derived_columns macro and only worked by chance.

This commit fixed some bugs with the derived columns, and since this commit your syntax should no longer work.

I am afraid that you need to update your syntax to be able to use the latest version of datavault4dbt..