Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/plugin dbt parser #75

Closed
wants to merge 123 commits into from
Closed

Feature/plugin dbt parser #75

wants to merge 123 commits into from

Conversation

kiranvasudev
Copy link
Contributor

No description provided.

siklosid and others added 30 commits September 30, 2023 15:45
@siklosid
Turning follow_external_dependency into bool or dict property
ea36004
@siklosid
Handling the new dict format of follow_external_dependency
5113752
@siklosid
Changing test case to see if it handles sensor parameters properly
fc2a2e2
@siklosid
Merge pull request #23 from chocoapp/feature/sensor_parameters
d0780dd 
Feature/sensor parameters
upgrade version of tenacity
db6dee2
@claudiazi
Merge pull request #24 from chocoapp/feature/DATA-1562-upgrade-tenacity
bd5adcf 
upgrade version of tenacity
@kiranvasudev
added new dbt config parser module
d463438
@kiranvasudev
added class with constructor
a91a58f
@kiranvasudev
added method to parse dbt model inputs
102620e
@kiranvasudev
added functions to generate dagger input and outputs for dbt models
c61e656
@kiranvasudev
added fn to generate io for dbt task
29858a9
@kiranvasudev
black format
1a93f05
@kiranvasudev
use new module in fn to generate configs
847f622
@kiranvasudev
changed output bucket
104013a
@kiranvasudev
fixed import
de82c14
@kiranvasudev
renamed functions and variables
dcb6854
@kiranvasudev
added type hints and docstrings
fb6f8f6
@kiranvasudev
changed how models are selected when config is generated
06dc86b
@kiranvasudev
black
19a13bc
@kiranvasudev
fix project dir to load the correct profile
5f83797
@kiranvasudev
get external_location from config instead of unrendered config
209efd6
@kiranvasudev
renamed variables for better understanding
3c4e4c3
@kiranvasudev
added doctest for fn
1bd525c
@kiranvasudev
added files for fixtures and tests
4b651ad
@kiranvasudev
added fixture for manifest and profiles
7123cbc
@kiranvasudev
setUp test class
77fb2b2
@kiranvasudev
added test for get_dbt_model_parents
47c0638
@kiranvasudev
added tests for generate_dagger_inputs
4eced6b
@kiranvasudev
added test for generate_dagger_outputs
aa767e8
@kiranvasudev
fixed type hint
8cecbe6
claudiazi and others added 29 commits April 29, 2024 14:09
fix: define the correct target_config for databricks adapter
7c1aa22
fix: _generate_dagger_output
40b28ef
extend: command
9c0311f
fix: _generate_command
14f9d1f
@claudiazi
Merge pull request #37 from chocoapp/feature/DATA-1811-databricks-io
0d3947f 
Feature/data 1811 databricks io
fix: _get_model_data_location in DatabricksDBTConfigParser
f7cc983
@claudiazi
Merge pull request #38 from chocoapp/feature/DATA-1811-databricks-io
1523648 
fix: _get_model_data_location in DatabricksDBTConfigParser
fix: generate_task for dbt tasks
59cf847
@claudiazi
Merge pull request #39 from chocoapp/feature/DATA-1811-databricks-io
a25ccc9 
fix: generate_task for dbt tasks
@siklosid
Replacing string replacement with jinja in module processor
c44a7b5
@siklosid
Merge pull request #40 from chocoapp/feature/DATA-1699_using_jinja_in…
e687686 
…_module_processor

Replacing string replacement with jinja in module processor
feat: adjust the dbt config parser so that view/ephemeral staging lay…
1bc6bf2 
…er doesnt need a task
feat: adjust io for the materalised staging model
22ca09b
feat: restructure the dbt dagger task input & output
f2f8015
@claudiazi
Merge pull request #41 from chocoapp/feat/DATA-1996-improve-dbt-parser
81e75da 
DATA-1996: adjust the dbt config parser so that view/ephemeral staging layer doesnt need a task
@siklosid
Module generation with generalised jinja parameters
698358d
@siklosid
Now it's possible to assign task to task groups
cd12dda
@siklosid
Adding default value to the parameter
0e6a6a6
@siklosid
Merge pull request #42 from chocoapp/feature/DATA-2002_generalise_jin…
f041aa8 
…ja_params

Feature/data 2002 generalise jinja params
@kiranvasudev
removed custom dbt task generation logic from Module
8d0e23f
@kiranvasudev
add plugins path to dagger config
77f6026
@kiranvasudev
added function to load plugins
fbb648b
@kiranvasudev
load plugins and render jinja
90bb16b
@kiranvasudev
iterate over multiple folders and their subfolders
1d09c57
@kiranvasudev
exclude all files starting with __
da0528a
@kiranvasudev
added plugin to dagger config
a8aeff7
@kiranvasudev
added logging for plugins
6addf98
@kiranvasudev
fix
b8feafb
@kiranvasudev
added tests for plugins
d8c6384
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kiranvasudev @siklosid @claudiazi