Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Wildcards node selection not working as expected. #11333

Open
2 tasks done
H-Max opened this issue Feb 25, 2025 · 1 comment
Open
2 tasks done

[Bug] Wildcards node selection not working as expected. #11333

H-Max opened this issue Feb 25, 2025 · 1 comment
Labels
bug Something isn't working triage

Comments

@H-Max
Copy link

H-Max commented Feb 25, 2025

Is this a new bug in dbt-core?

  • I believe this is a new bug in dbt-core
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

There seems to be issues with selecting nodes to run when using wildcars.

I have a model name "INT_OH_TOPAZ_PER" and I try to select it (in dbt run / dbt list, dbt whatever) and the following behavior does not seem to be correct:

  • dbt list -s "INT_OH_TOPAZ_PER" works
  • dbt list -s "INT_OH_TOPAZ_*" does not work (it should)
  • dbt list -s "*_OH_TOPAZ_*" works
  • dbt list -s "INT_OH_TOPAZ_PE?" does not work (it should)
  • dbt list -s "*INT_OH_TOPAZ_PE?" works (it should)

It seems that starting the selection string with a * is mandatory in order for further wildcards to work.

The documentation does not mention this behavior : https://docs.getdbt.com/reference/node-selection/methods

(if it's the expected behavior, then it should be explicitely written in the doc)

Expected Behavior

We should be able to use wildcards without having to start the selector string with "*"

Steps To Reproduce

Create a model and try to select the model in various dbt CLI commands with wildcards in it, following the examples in the behavior description above.

Relevant log output

Environment

- OS: MacOS
- Python: 3.11.x
- dbt: 1.9.2 (core) 1.9.0 (BigQuery)

Which database adapter are you using with dbt?

bigquery

Additional Context

No response

@H-Max H-Max added bug Something isn't working triage labels Feb 25, 2025
@mouradgh
Copy link

mouradgh commented Mar 2, 2025

Here is Claude Code's suggested solution, if someone wants to implement it and test it :

Diagnostic :

I found the issue with wildcard selection patterns in dbt. Looking at the code, particularly in selector_methods.py in the
  is_selected_node function around line 69:

  The problem lies in how wildcards are processed in the node selection logic. When a selection pattern doesn't start with a wildcard
  (*), the code processes it character by character until it finds a wildcard. If no wildcard is found at the beginning, it tries to
  match the initial parts exactly, and only after finding a wildcard does it switch to pattern matching with fnmatch().

  The specific issue is in the is_selected_node function (lines 69-108) which:
  1. First checks for exact matches (line 79)
  2. Then checks each component before finding a wildcard (lines 88-95)
  3. Only switches to wildcard matching mode when it encounters a wildcard (lines 97-105)

  This is why patterns that don't start with wildcards but contain them later don't work as expected - the selector checks for exact
  matches in the beginning parts and fails before it reaches the wildcard.

Solution :

I've modified the is_selected_node function to allow wildcard patterns to work anywhere in the selector string. Here's what changed:

  1. The function now checks if the selector string contains any wildcard characters (*, ?, [, ]) at the beginning.
  2. If wildcards are present anywhere in the selector:
    - For regular models, it uses fnmatch to compare the model name directly against the entire selector pattern
    - For versioned models, it matches against the combined versioned model name
  3. If no wildcards are present, it continues with the original dot-notation logic for exact matches.

  This change should fix your reported issues:
  - dbt list -s "INT_OH_TOPAZ_*" will now work correctly, matching any model whose name starts with "INT_OH_TOPAZ_"
  - dbt list -s "INT_OH_TOPAZ_PE?" will now work correctly, matching models whose names match that pattern

  The change focuses on making the wildcard pattern matching work intuitively regardless of where the wildcard characters appear in
  the selector. You no longer need to prefix your selectors with * to make wildcard patterns work.

Code (replaces lines 87 through 105) :

    # Check if selector contains any wildcards
    has_wildcards = any(wildcard in node_selector for wildcard in ("*", "?", "[", "]"))

    # If the selector has wildcards, use fnmatch for the entire string
    if has_wildcards:
        # For selectors with wildcards, use fnmatch directly on the model name (last part of fqn)
        # This allows wildcards to work regardless of position
        if is_versioned:
            # For versioned models, match against the last two parts
            return fnmatch("_".join(fqn[-2:]), node_selector)
        else:
            # For regular models, match against the model name
            return fnmatch(fqn[-1], node_selector)

    # For dot-notation selectors without wildcards, continue with the existing logic
    selector_parts = node_selector.split(".")
    for i, selector_part in enumerate(selector_parts):
        if i >= len(flat_fqn) or flat_fqn[i] != selector_part:
            return False

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working triage
Projects
None yet
Development

No branches or pull requests

2 participants