Skip to content

[Feature] dbt ingestion: skip compiled-only test results that never executed SQL #26785

@nr-mdakilahmed

Description

@nr-mdakilahmed

Is your feature request related to a problem

When multiple dbt Cloud jobs share the same project, a test node can appear in run_results.json from a job that only runs dbt run (models only). dbt includes these tests as compiled-only entries — no SQL is executed — but marks them status: "success". The OM connector ingests this as a real TestCaseStatus.Success, producing phantom test results.

This causes:

  • Duplicate results — the same test gets two entries per day (one real from the dbt build job, one phantom from the dbt run job)
  • Duplicate alerts — each result triggers a separate change event and notification
  • Misleading test history — the timeseries graph shows conflicting statuses (e.g. Success + Aborted for the same test)

How to identify compiled-only entries

Field Real execution Compiled-only
failures 0, 1, N (integer) null
message "Pass", "Got 1 result..." null
adapter_response {query_id, rows_affected, ...} {}
execution_time Seconds < 0.1s

Describe the solution you'd like

In add_dbt_test_result() (metadata.py#L1036), skip entries where no SQL was executed:

# After fetching dbt_test_result, before status mapping:
if dbt_test_result.failures is None and not dbt_test_result.message:
    logger.info(
        f"Skipping compiled-only test result for node: {manifest_node.name}"
    )
    return

A real test always populates failures (even 0 on pass) and message. Both being null is a reliable signal that the test was only compiled.

Additional context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    No status

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions