fix(pythonic-resources): ResourceDependency with default causes RecursionError#33668
Conversation
Fixes dagster-io#33650. When a `ConfigurableResource` subclass instance is used as a default value for a `ResourceDependency` field: class Outer(ConfigurableResource): inner: ResourceDependency[Inner] = Inner() The `Inner()` instance was stored as a class attribute of `Outer`. Because `ConfigurableResourceFactory` inherits `__get__`/`__set__` from `TypecheckAllowPartialResourceInitParams`, every `ConfigurableResource` instance is a Python *data descriptor*. Accessing `outer.inner` therefore triggered `TypecheckAllowPartialResourceInitParams.__get__`, which called `getattr(obj, self._assigned_name)`, which re-triggered `__get__` ad infinitum. Two changes fix this: 1. **`typing_utils.py` – `TypecheckAllowPartialResourceInitParams`** - `__get__`: when `obj is None` (class-level access) return `self` so Pydantic can read the default correctly. When `obj` is an instance, read from `obj.__dict__` directly to bypass the descriptor protocol and break the recursion. - `__set__`: use `object.__setattr__` instead of `setattr` so that writing a value to an instance attribute also bypasses the descriptor protocol. 2. **`resource.py` – `ConfigurableResourceFactory.__init__`** After `super().__init__()` Pydantic has populated the instance `__dict__` with all field values (including defaults it can now read thanks to fix 1). Re-running `separate_resource_params` over `self.__dict__` picks up any resource-typed fields populated from defaults, ensuring they are tracked in `_nested_resources` and correctly resolved at execution time. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Greptile SummaryThis PR fixes a Changes:
Confidence Score: 5/5
|
| Filename | Overview |
|---|---|
| python_modules/dagster/dagster/_config/pythonic_config/typing_utils.py | Fixed recursive descriptor loop: __get__ now returns self on class-level access and reads from obj.__dict__ to bypass the protocol. __set__ uses object.__setattr__, but contains a dead-code else branch. |
| python_modules/dagster/dagster/_config/pythonic_config/resource.py | Post-super().__init__() re-scan of self.__dict__ correctly picks up resource-typed fields populated from class-level defaults; merge order {**all_resource_pointers, **resource_pointers} properly preserves explicit overrides. Minor extra blank line. |
| python_modules/dagster/dagster_tests/core_tests/resource_tests/pythonic_resources/test_nesting.py | Two new regression tests covering sensor context and asset execution paths for the fix; solid coverage of default, explicit-override, and nested-resource-tracking scenarios. Minor redundancy between Pattern 1 and Pattern 4. |
Sequence Diagram
sequenceDiagram
participant User
participant OuterResource
participant Pydantic
participant TypecheckDescriptor as TypecheckAllowPartialResourceInitParams.__get__
participant InstanceDict as obj.__dict__
Note over User,InstanceDict: BEFORE fix — RecursionError
User->>OuterResource: outer.inner
OuterResource->>TypecheckDescriptor: __get__(obj=outer, owner=Outer)
TypecheckDescriptor->>TypecheckDescriptor: getattr(outer, "inner")
TypecheckDescriptor->>TypecheckDescriptor: __get__(obj=outer, owner=Outer)
TypecheckDescriptor-->>TypecheckDescriptor: ∞ RecursionError
Note over User,InstanceDict: AFTER fix — correct resolution
User->>OuterResource: outer.inner
OuterResource->>TypecheckDescriptor: __get__(obj=outer, owner=Outer)
TypecheckDescriptor->>InstanceDict: obj.__dict__["inner"]
InstanceDict-->>TypecheckDescriptor: InnerResource instance
TypecheckDescriptor-->>User: InnerResource instance
Note over User,InstanceDict: Class-level access (Pydantic reading default)
Pydantic->>TypecheckDescriptor: __get__(obj=None, owner=Outer)
TypecheckDescriptor-->>Pydantic: self (descriptor/default InnerResource instance)
Reviews (1): Last reviewed commit: "fix: ResourceDependency with default no ..." | Re-trigger Greptile
| if obj is not None: | ||
| object.__setattr__(obj, self._assigned_name, value) | ||
| else: | ||
| setattr(obj, self._assigned_name, value) |
There was a problem hiding this comment.
Dead-code
else branch in __set__
Python's descriptor protocol never calls __set__ with obj=None — __set__ is only invoked on instances, not via class-level attribute assignment (which would replace the descriptor in the class __dict__ directly). The else branch therefore can never be reached, and if it somehow were, setattr(None, …) would raise TypeError.
Consider simplifying to just the guard:
| if obj is not None: | |
| object.__setattr__(obj, self._assigned_name, value) | |
| else: | |
| setattr(obj, self._assigned_name, value) | |
| if obj is not None: | |
| object.__setattr__(obj, self._assigned_name, value) |
| return out | ||
|
|
||
|
|
||
|
|
There was a problem hiding this comment.
Extra blank line between top-level functions
Three blank lines are present between separate_resource_params and _call_resource_fn_with_default; PEP 8 recommends two.
| return out | |
| return out | |
| def _call_resource_fn_with_default( |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| # Pattern 4: sensor resolves the nested resource correctly end-to-end | ||
| # (access via context.resources to verify full resource initialization path) | ||
| with dg.build_sensor_context( | ||
| resources={"outer": OuterResource()}, sensor_name="test_sensor_33650" | ||
| ) as ctx: | ||
| outer = ctx.resources.outer | ||
| assert outer.greet() == "hello" |
There was a problem hiding this comment.
"Pattern 4" is identical to "Pattern 1" — both create a build_sensor_context with OuterResource() and assert outer.greet() == "hello". Consider replacing with a distinct scenario (e.g. verifying the sensor body runs via list(my_sensor(ctx))) or removing it to reduce noise.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Summary
Fixes #33650.
When a
ConfigurableResourcesubclass instance is used as a class-level default for aResourceDependencyfield, accessing that field triggered infinite recursion:Root cause:
ConfigurableResourceFactoryinherits__get__/__set__fromTypecheckAllowPartialResourceInitParams, making everyConfigurableResourceinstance a Python data descriptor. WhenInner()is stored as a class attribute ofOuter, accessingouter.innercalledTypecheckAllowPartialResourceInitParams.__get__which calledgetattr(obj, self._assigned_name), which re-triggered__get__— infinite recursion.A secondary effect was that Pydantic could not read the
Inner()class-level default (becauseInner().__get__(None, Outer)raisedAttributeErrorbefore the fix), causing_nested_resourcesto be empty and the nested resource's lifecycle to be skipped.Fix — two changes:
TypecheckAllowPartialResourceInitParams.__get__/__set__(typing_utils.py):__get__: returnselfwhenobj is None(class-level access) so Pydantic can read the default. Whenobjis an instance, read fromobj.__dict__directly to bypass the descriptor protocol.__set__: useobject.__setattr__instead ofsetattrto avoid re-entering the descriptor.ConfigurableResourceFactory.__init__(resource.py):After
super().__init__(), Pydantic has populatedself.__dict__with all values including defaults. Re-runseparate_resource_paramsoverself.__dict__to pick up resource-typed fields populated from defaults, ensuring they are tracked in_nested_resourcesand resolved at execution time.Test plan
test_resource_dependency_with_default_no_recursioncovers sensor contexttest_resource_dependency_with_default_in_assetcovers asset execution🤖 Generated with Claude Code