-
Notifications
You must be signed in to change notification settings - Fork 1.7k
[Feature] dbt ingestion: skip compiled-only test results that never executed SQL #26785
Copy link
Copy link
Open
Description
Is your feature request related to a problem
When multiple dbt Cloud jobs share the same project, a test node can appear in run_results.json from a job that only runs dbt run (models only). dbt includes these tests as compiled-only entries — no SQL is executed — but marks them status: "success". The OM connector ingests this as a real TestCaseStatus.Success, producing phantom test results.
This causes:
- Duplicate results — the same test gets two entries per day (one real from the
dbt buildjob, one phantom from thedbt runjob) - Duplicate alerts — each result triggers a separate change event and notification
- Misleading test history — the timeseries graph shows conflicting statuses (e.g. Success + Aborted for the same test)
How to identify compiled-only entries
| Field | Real execution | Compiled-only |
|---|---|---|
failures |
0, 1, N (integer) |
null |
message |
"Pass", "Got 1 result..." |
null |
adapter_response |
{query_id, rows_affected, ...} |
{} |
execution_time |
Seconds | < 0.1s |
Describe the solution you'd like
In add_dbt_test_result() (metadata.py#L1036), skip entries where no SQL was executed:
# After fetching dbt_test_result, before status mapping:
if dbt_test_result.failures is None and not dbt_test_result.message:
logger.info(
f"Skipping compiled-only test result for node: {manifest_node.name}"
)
returnA real test always populates failures (even 0 on pass) and message. Both being null is a reliable signal that the test was only compiled.
Additional context
- OM version: 1.11.8
- dbt Cloud connector, dbt Core 1.7+
- Related: [Feature] dbt connector: fall back to test_metadata.kwargs.column_name when top-level column_name is null #26606
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
No status