dbt-core v1.0.5rc1

dbt-core 1.0.5-rc1 - March 21, 2022

Fixes

• Fix bug causing empty node level meta, snapshot config errors (#4459, #4726)
• Support click versions in the v7.x series (#4566, #4681)
• Fixed a bug where nodes that depend on multiple macros couldn't be selected using -s state:modified (#4678, #4820)
• Catch all Requests Exceptions on deps install to attempt retries. Also log the exceptions hit. (#4849, #4865)
• Fix partial parsing bug with multiple snapshot blocks (#4771, #4773)
• Use cli_vars instead of context to create package and selector renderers (#4876, #4878)
• depend on new dbt-extractor version with fixed github links (#4891, #4890)

Under the Hood

• Automate changelog generation with changie (#4652, #4743)
• Fix broken links for changelog generation and tweak GHA to only post a comment once when changelog entry is missing. (#4848, #4857)

Docs

• Resolve errors related to operations preventing DAG from generating in the docs. Also patch a spark issue to allow search to filter accurately past the missing columns. (#4578, #4763)
  Contributors:
  • twilly (#4681)

dbt-core v1.0.4

This release contains one change, intended to quickly resolve failing installations of dbt-core for some users.

Fixes

• Depend on new dbt-extractor version with fixed GitHub links to resolve Homebrew installation issues (#4891, #4890)

dbt-core v1.0.3

Full changelog

Fixes

• Fix bug accessing target fields in deps and clean commands. This was a net-new bug in v1.0.2 that would affect a fraction of active projects after upgrading (#4752, #4758)

dbt-core v1.0.2

Release notes

dbt-core v1.0.2rc1

Release notes

dbt-core v1.0.1

This is the final release for v1.0.1 and includes only changes that were introduced in the RC

Fixes

• Fix wrong url in the dbt docs overview homepage (#4442)
• Fix redefined status param of SQLQueryStatus to typecheck the string which passes on ._message value of AdapterResponse or the str value sent by adapter plugin. (#4463)
• Fix DepsStartPackageInstall event to use package name instead of version number. (#4482)
• Reimplement log message to use adapter name instead of the object method. (#4501)
• Issue better error message for incompatible schemas (#4470, #4497)
• Remove secrets from error related to packages. (#4507)
• Prevent coercion of boolean values (True, False) to numeric values (0, 1) in query results (#4511, #4512)
• Fix error with an env_var in a project hook (#4523, #4524)

Docs

• Fix missing data on exposures in docs (#4467)

Contributors:

• remoyson (#4442)

dbt-core v1.0.1rc1

A first release candidate for dbt-core v1.0.1

Fixes

• Fix wrong url in the dbt docs overview homepage (#4442)
• Fix redefined status param of SQLQueryStatus to typecheck the string which passes on ._message value of AdapterResponse or the str value sent by adapter plugin. (#4463)
• Fix DepsStartPackageInstall event to use package name instead of version number. (#4482)
• Reimplement log message to use adapter name instead of the object method. (#4501)
• Issue better error message for incompatible schemas (#4470, #4497)
• Remove secrets from error related to packages. (#4507)
• Prevent coercion of boolean values (True, False) to numeric values (0, 1) in query results (#4511, #4512)
• Fix error with an env_var in a project hook (#4523, #4524)

Docs

• Fix missing data on exposures in docs (#4467)

Contributors:

• remoyson (#4442)

dbt-core v1.0.0

dbt Core 1.0.0 - W. E. B. Du Bois (December 3, 2021)

• 💬 Discourse: v1.0
• 📖 New & changed documentation

Breaking changes

• The two type of test definitions are now "singular" and "generic" (instead of "data" and "schema", respectively). The test_type: selection method accepts test_type:singular and test_type:generic. (It will also accept test_type:schema and test_type:data for backwards compatibility) (#3234, #3880). Not backwards compatible: The --data and --schema flags to dbt test are no longer supported, and tests no longer have the tags 'data' and 'schema' automatically applied.
• Deprecated the use of the packages arg adapter.dispatch in favor of the macro_namespace arg. (#3895)
• Enable on-run-start and on-run-end hooks for dbt test. Add flags.WHICH to execution context, representing current task (#3463, #4004)
• Replace greedy flag/property for test selection with indirect_selection: eager/cautious flag/property. Set to eager by default. Note: This reverts test selection to its pre-v0.20 behavior by default. dbt test -s my_model will select multi-parent tests, such as relationships, that depend on unselected resources. To achieve the behavior change in v0.20 + v0.21, set --indirect-selection=cautious on the CLI or indirect_selection: cautious in yaml selectors. (#4082, #4104)
• In v1.0.0, pip install dbt will raise an explicit error. Instead, please use pip install dbt-<adapter> (to use dbt with that database adapter), or pip install dbt-core (for core functionality). For parity with the previous behavior of pip install dbt, you can use: pip install dbt-core dbt-postgres dbt-redshift dbt-snowflake dbt-bigquery (#4100, #4133)
• Reorganize the global_project (macros) into smaller files with clearer names. Remove unused global macros: column_list, column_list_for_create_table, incremental_upsert (#4154)
• Introduce structured event interface, and begin conversion of all legacy logging (#3359, #4055)
  • This is a breaking change for adapter plugins, requiring a very simple migration. See events module README for details.
  • If you maintain another kind of dbt-core plugin that makes heavy use of legacy logging, and you need time to cut over to the new event interface, you can re-enable the legacy logger via an environment variable shim, DBT_ENABLE_LEGACY_LOGGER=True. Be advised that we will remove this capability in a future version of dbt-core.
• Restrict secret env vars (prefixed DBT_ENV_SECRET_) to profiles.yml + packages.yml only. Raise an exception if a secret env var is used elsewhere (#4310, #4311)
• Reorder arguments to config.get() so that default is second (#4273, #4297)

Features

• Normalize global CLI arguments/flags (#2990, #3839)
• Turns on the static parser by default and adds the flag --no-static-parser to disable it. (#3377, #3939)
• Generic test FQNs have changed to include the relative path, resource, and column (if applicable) where they are defined. This makes it easier to configure them from the tests block in dbt_project.yml (#3259, #3880
• Turn on partial parsing by default (#3867, #3989)
• Generic test can now be added under a generic subfolder in the test-paths directory. (#4052)
• Add result:<status> selectors to automatically rerun failed tests and erroneous models. This makes it easier to rerun failed dbt jobs with a simple selector flag instead of restarting from the beginning or manually running the dbt models in scope. (#3859, #4017)
• dbt init is now interactive, generating profiles.yml when run inside existing project (#3625)
• Avoid error when missing column in YAML description (#4151, #4285)
• Allow --defer flag to dbt snapshot (#4110, #4296)
• Install prerelease packages when version explicitly references a prerelease version, regardless of install-prerelease status (#4243, #4295)
• Add data attributes to json log messages (#4301)
• Add event codes to all log events (#4319)
• Allow nullable error_after in source freshness (#3874, #3955)
• Add metrics nodes (#4071, #4235)
• Add support for dbt init <project_name>, and support for skip_profile_setup argument (dbt init -s) (#4156, #4249)

Fixes

• Add generic tests defined on sources to the manifest once, not twice (#3347, #3880)
• Skip partial parsing if certain macros have changed (#3810, #3982)
• Enable cataloging of unlogged Postgres tables (3961, #3993)
• Fix multiple disabled nodes (#4013, #4018)
• Fix multiple partial parsing errors (#3996, #4020)
• Return an error instead of a warning when runing with --warn-error and no models are selected (#4006, #4019)
• Fixed bug with error_if test option (#4070)
• Allow retries when the answer from a dbt deps is None (#4178, #4225)
• Allow specifying default in Jinja config.get with default keyword (#4273, #4297)
• Fix serialization error with missing quotes in metrics model ref (#4252, #4287)
• Correct definition of 'created_at' in ParsedMetric nodes (#4298, #4299)
• Fix serialization error with missing quotes in metrics model ref (#4252, #4287)
• Correct definition of 'created_at' in ParsedMetric nodes (#4298, #4299)
• Support partial parsing of env_vars in metrics (#4253, #4322)
• Fix typo in UnparsedSourceDefinition.__post_serialize__ (#3545, #4349)
• Configure the CLI logger destination to use stdout instead of stderr (#4368)
• Make the size of EVENT_HISTORY configurable, via EVENT_BUFFER_SIZE global config (#4411, #4416)
• Change type of log_format in profiles.yml user config to be string, not boolean (#4394)

Docs

• Fix non-alphabetical sort of Source Tables in source overview page (docs#81, docs#218)
• Add title tag to node elements in tree (docs#202, docs#203)
• Account for test rename: schema → generic, ...

dbt Core 1.0.0rc3

A third release candidate for dbt-core v1.0.0. Changes since rc2:

Fixes

• Support partial parsing of env_vars in metrics (#4253, #4322)
• Fix typo in UnparsedSourceDefinition.__post_serialize__ (#3545, #4349)

Under the hood

• Change some CompilationExceptions to ParsingExceptions (#4254, #4328)
• Reorder logic for static parser sampling to speed up model parsing (#4332)
• Use more augmented assignment statements (#4315), (#4311)
• Adjust logic when finding approximate matches for models and tests (#3835), #4076)
• Restore small previous behaviors for logging: JSON formatting for first few events; WARN-level stdout for list task; include tracking events in dbt.log (#4341)

Contributors:

• @sarah-weatherbee (#4331)
• @emilieschario (#4076)
• @sneznaj (#4349)

dbt 0.21.1

Fixes

• Performance: Use child_map to find tests for nodes in resolve_graph (#4012, #4057)
• Switch unique_field from abstractproperty to optional property. Add docstring (#4025, #4028)
• Fix multiple partial parsing errors (#3996, #4020)
• Include only relational nodes in database_schema_set (#4063, #4077)
• Fixed bug with error_if test option (#4070)
• Added support for tests on databases that lack real boolean types. (#4084)
• Prefer macros defined in the project over the ones in a package by default (#4106, #4114)
• Scrub secrets coming from CommandErrors so they don't get exposed in logs. (#4138)
• Syntax fix in alter_relation_add_remove_columns if only removing columns in on_schema_change: sync_all_columns (#4147)
• Increase performance of graph subset selection (#4135,#4155)
• Add downstream test edges for build task only. Restore previous graph construction, compilation performance, and node selection behavior (test+) for all other tasks (#4135, #4143)
• Don't require a strict/proper subset when adding testing edges to specialized graph for build (#4158, #4158)
• Capping google-api-core to version 1.31.3 due to protobuf dependency conflict (#4192)
• Add get_where_subquery to test macro namespace, fixing custom generic tests that rely on introspecting the model arg at parse time (#4195, #4197)

Contributors

• @ljhopkins2 (#4077)
• @JCZuurmond (#4114)