Skip to content

feat: enforce ENABLED_BROKERS and DEFAULT_BROKER at runtime#1007

Open
jwesleye wants to merge 2 commits into
mainfrom
agentshore/991-config-enabled-brokers-default-broker-env-vars-ignored
Open

feat: enforce ENABLED_BROKERS and DEFAULT_BROKER at runtime#1007
jwesleye wants to merge 2 commits into
mainfrom
agentshore/991-config-enabled-brokers-default-broker-env-vars-ignored

Conversation

@jwesleye

Copy link
Copy Markdown
Collaborator

Closes #991

Changes

  • Add src/open_stocks_mcp/server/broker_filter.py — a new module following the same call_tool-wrapping pattern as tool_execution_limits.py. It provides:
    • _tool_broker(name) — maps any tool name to its required broker ("schwab" for names containing "schwab", None for meta/cross-broker tools, "robinhood" for everything else)
    • install_broker_filter(mcp_server, enabled_brokers) — wraps call_tool to return a structured error for disabled-broker tools, and wraps list_tools to omit them from the listing
  • Wire install_broker_filter into main() in app.py, called once after setup_brokers() completes, just before the transport starts accepting connections
  • Add 17 unit tests in tests/unit/test_broker_filter.py covering mapping, blocking, list-filtering, and idempotency

Behaviour

Scenario Before After
ENABLED_BROKERS=robinhood (default), call schwab_quote Executes (returns auth error or data) Returns structured broker disabled error
ENABLED_BROKERS=robinhood, call list_tools Lists all 152 tools Lists Robinhood + meta tools only
ENABLED_BROKERS=robinhood,schwab All tools All tools
Meta tools (broker_status, list_brokers, etc.) Always accessible Always accessible

Test plan

  • uv run pytest tests/unit/test_broker_filter.py -v — 17 unit tests, all pass
  • uv run pytest -m "not slow and not exception_test and not integration and not rate_limited and not performance" -q — 1479 passed
  • uv run ruff check . && uv run ruff format . && uv run mypy src/ — all clean

Install a broker-availability filter on the MCP server's call_tool and
list_tools dispatch so tools for disabled brokers are blocked from
execution and hidden from the tool listing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@claude

claude Bot commented May 28, 2026

Copy link
Copy Markdown

Claude finished @jwesleye's task —— View job


Code Review Complete ✅

I'll provide comprehensive feedback on code quality, potential issues, performance considerations, security concerns, and test coverage.

Todo List

  • Read project guidelines from CLAUDE.md
  • Analyze broker_filter.py implementation
  • Review app.py integration changes
  • Examine test coverage and quality
  • Check for potential bugs and security issues
  • Provide comprehensive feedback

Review Summary

Overall Assessment: APPROVED with minor suggestions

This is a well-implemented feature that cleanly enforces broker filtering at runtime. The code follows the existing patterns, has excellent test coverage, and addresses a clear functional requirement.

✅ Strengths

Architecture & Design

  • Follows existing patterns: Mirrors tool_execution_limits.py approach perfectly - consistent with codebase architecture
  • Clean separation of concerns: _tool_broker() mapping logic separated from filter installation
  • Idempotent design: Re-installation updates enabled brokers without wrapper stacking
  • Non-invasive: Uses attribute-based wrapper detection to avoid conflicts

Code Quality

  • Excellent type hints: Full type annotations including frozenset[str] and proper return types
  • Clear naming: _BROKER_AGNOSTIC_TOOLS, install_broker_filter() are self-documenting
  • Good error messages: Structured error responses with actionable guidance
  • Proper async patterns: Correctly handles FastMCP's async tool interface

Test Coverage

  • Comprehensive coverage: 17 tests covering all major scenarios
  • Well-structured tests: Clear test classes by functionality area
  • Edge case coverage: Tests idempotency, wrapper stacking, and error scenarios
  • Good helper functions: _make_server(), _is_blocked() improve test readability

🔍 Code Review Details

broker_filter.py:36-48 - Broker Mapping Logic

def _tool_broker(tool_name: str) -> str | None:
    if tool_name in _BROKER_AGNOSTIC_TOOLS:
        return None
    if "schwab" in tool_name:
        return "schwab"
    return "robinhood"

Well-designed: Simple, predictable logic that's easy to extend for future brokers.

broker_filter.py:61-63 - Idempotency Implementation

if getattr(mcp_server, _WRAPPER_ATTR, False):
    setattr(mcp_server, _ENABLED_ATTR, list(enabled_brokers))
    return

Smart approach: Updates enabled broker list without re-wrapping, preventing performance degradation.

broker_filter.py:82-87 - Error Response Format

return mcp.types.CallToolResult(
    content=[mcp.types.TextContent(type="text", text=json.dumps(error_data))],
    isError=True,
)

Consistent: Follows MCP standards and project's JSON response pattern.

app.py:2304 - Integration Point

install_broker_filter(server, config.brokers.enabled_brokers)

Perfect placement: Called after setup_brokers() but before transport starts - exactly right timing.

🛡️ Security Assessment

  • No security vulnerabilities identified
  • Input validation: Environment variables properly parsed and validated
  • Error handling: No sensitive information leaked in error messages
  • Access control: Proper enforcement of broker-level access restrictions

⚡ Performance Considerations

  • Minimal overhead: Simple string matching and attribute lookups
  • No wrapper stacking: Idempotent design prevents performance degradation
  • Efficient filtering: frozenset lookup for agnostic tools is O(1)
  • Lazy evaluation: No unnecessary work when brokers are enabled

📝 Minor Suggestions

1. Documentation Enhancement (Optional)

Consider adding a brief docstring example to _tool_broker():

def _tool_broker(tool_name: str) -> str | None:
    """Return the broker name required by *tool_name*, or ``None`` if agnostic.
    
    Examples:
        >>> _tool_broker("schwab_quote")
        "schwab"
        >>> _tool_broker("portfolio") 
        "robinhood"
        >>> _tool_broker("broker_status")
        None
    """

2. Future-Proofing (Optional)

The substring matching for Schwab ("schwab" in tool_name) works well now but consider if you want to be more explicit for future brokers. Current approach is fine given the existing naming conventions.

🧪 Test Quality Assessment

Excellent test suite with these highlights:

  • Journey markers: Proper @pytest.mark.journey_system usage
  • Async testing: Correct use of @pytest.mark.asyncio
  • Comprehensive scenarios: Tests both call_tool and list_tools behavior
  • Clear test structure: Well-organized classes by functionality
  • Good assertions: Tests both positive and negative cases thoroughly

🚀 Integration Assessment

Perfect integration approach:

  • Timing: Filter installed after broker setup, before transport start
  • Configuration: Uses centralized config.brokers.enabled_brokers
  • Non-breaking: Existing functionality unchanged when all brokers enabled
  • Graceful degradation: Clear error messages when brokers disabled

Final Recommendation

✅ APPROVE - This PR is ready for merge. It's a clean, well-tested implementation that follows the project's patterns and successfully addresses the issue requirements. The code quality is high, test coverage is comprehensive, and the integration is seamless.


@unseriousAI unseriousAI left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AGENTSHORE_CODE_REVIEW
head_sha: 2fcc70a
status: BLOCK
spec_compliance: BLOCK
blocking_findings: 1
non_blocking_findings: 0

Summary

REQUEST_CHANGES: issue #991 requires runtime enforcement for both ENABLED_BROKERS and DEFAULT_BROKER, but this PR only enforces ENABLED_BROKERS.

Blocking

  1. src/open_stocks_mcp/server/broker_filter.py:34-46, 49-101; src/open_stocks_mcp/server/app.py:2300-2304 — Spec mismatch with issue #991. The issue explicitly requires runtime enforcement of both ENABLED_BROKERS and DEFAULT_BROKER, and says these values should gate which broker tools are registered and active at server start. The implementation adds dispatch-time filtering for enabled brokers only; there is no logic that reads or enforces DEFAULT_BROKER behavior, and no tests covering DEFAULT_BROKER outcomes. Suggested fix: define and implement DEFAULT_BROKER runtime semantics (for example, default routing for broker-agnostic calls and/or startup registration policy), then add focused unit/integration tests proving that behavior.

Suggestions

None

@unseriousAI

Copy link
Copy Markdown
Collaborator

Addressed the blocking review feedback around runtime DEFAULT_BROKER enforcement.

What changed:

  • Updated broker filter runtime mapping to honor configured DEFAULT_BROKER for non-Schwab/non-agnostic tools instead of hardcoded robinhood fallback.
  • Kept Schwab-prefix and broker-agnostic behavior unchanged.
  • Wired DEFAULT_BROKER through app startup when installing the runtime broker filter.
  • Added focused unit coverage for:
    • default-broker-based blocking/allowing for generic tools,
    • list_tools visibility under default-broker constraints,
    • idempotent reinstall updating default broker.

Validation evidence:

  • ============================= test session starts ==============================
    platform darwin -- Python 3.11.14, pytest-9.0.3, pluggy-1.6.0
    benchmark: 5.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
    rootdir: /Users/wes/Development/Open Agent Tools/agentshore-worktrees/open-stocks-mcp/agentshore-991-config-enabled-brokers-default-broker-env-vars-ignored
    configfile: pyproject.toml
    plugins: benchmark-5.2.3, mock-3.15.1, cov-7.1.0, timeout-2.4.0, asyncio-1.3.0, anyio-4.13.0
    timeout: 60.0s
    timeout method: thread
    timeout func_only: False
    asyncio: mode=Mode.AUTO, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
    collected 21 items

tests/unit/test_broker_filter.py ..................... [100%]

=============================== warnings summary ===============================
.venv/lib/python3.11/site-packages/_pytest/config/init.py:858
/Users/wes/Development/Open Agent Tools/agentshore-worktrees/open-stocks-mcp/agentshore-991-config-enabled-brokers-default-broker-env-vars-ignored/.venv/lib/python3.11/site-packages/_pytest/config/init.py:858: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.integration.live_market_harness
self.import_plugin(import_spec)

.venv/lib/python3.11/site-packages/websockets/legacy/init.py:6
/Users/wes/Development/Open Agent Tools/agentshore-worktrees/open-stocks-mcp/agentshore-991-config-enabled-brokers-default-broker-env-vars-ignored/.venv/lib/python3.11/site-packages/websockets/legacy/init.py:6: DeprecationWarning: websockets.legacy is deprecated; see https://websockets.readthedocs.io/en/stable/howto/upgrade.html for upgrade instructions
warnings.warn( # deprecated in 14.0 - 2024-11-09

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================== 21 passed, 2 warnings in 1.65s ========================

  • Exit code: 0
  • Result: 21 passed

@unseriousAI unseriousAI dismissed their stale review May 28, 2026 21:00

Addressed in 244851c: implemented DEFAULT_BROKER-aware runtime broker filter and added focused tests

@claude

claude Bot commented May 28, 2026

Copy link
Copy Markdown

Claude finished @jwesleye's task —— View job


Code Review Complete ✅

I'll provide comprehensive feedback on code quality, potential issues, performance considerations, security concerns, and test coverage.

Todo List

  • Read project guidelines from CLAUDE.md
  • Analyze broker_filter.py implementation
  • Review app.py integration changes
  • Examine test coverage and quality
  • Check for potential bugs and security issues
  • Provide comprehensive feedback

Review Summary

Overall Assessment: APPROVED with minor suggestions

This is a well-implemented feature that cleanly enforces broker filtering at runtime. The code follows the existing patterns, has excellent test coverage, and successfully addresses both ENABLED_BROKERS and DEFAULT_BROKER enforcement requirements from issue #991.

✅ Strengths

Architecture & Design

  • Follows existing patterns: Mirrors tool_execution_limits.py approach perfectly - consistent with codebase architecture
  • Clean separation of concerns: _tool_broker() mapping logic separated from filter installation
  • Idempotent design: Re-installation updates enabled brokers without wrapper stacking
  • Non-invasive: Uses attribute-based wrapper detection to avoid conflicts
  • DEFAULT_BROKER enforcement: Properly honors configured default broker for generic tools

Code Quality

  • Excellent type hints: Full type annotations including frozenset[str] and proper return types
  • Clear naming: _BROKER_AGNOSTIC_TOOLS, install_broker_filter() are self-documenting
  • Good error messages: Structured error responses with actionable guidance
  • Proper async patterns: Correctly handles FastMCP's async tool interface
  • Input validation: Normalizes default broker with fallback to "robinhood"

Test Coverage

  • Comprehensive coverage: 21 tests covering all major scenarios including new DEFAULT_BROKER logic
  • Well-structured tests: Clear test classes by functionality area
  • Edge case coverage: Tests idempotency, wrapper stacking, and error scenarios
  • Good helper functions: _make_server(), _is_blocked() improve test readability
  • DEFAULT_BROKER testing: Thorough coverage of default broker blocking/allowing behavior

🔍 Code Review Details

broker_filter.py:37-49 - Enhanced Broker Mapping Logic

def _tool_broker(tool_name: str, default_broker: str) -> str | None:
    if tool_name in _BROKER_AGNOSTIC_TOOLS:
        return None
    if "schwab" in tool_name:
        return "schwab"
    return default_broker

Excellent enhancement: Now properly honors configured default_broker instead of hardcoded "robinhood" fallback. This addresses the blocking review feedback perfectly.

broker_filter.py:64-66 - Default Broker Normalization

normalized_default = (default_broker or "robinhood").strip().lower()
if normalized_default not in {"robinhood", "schwab"}:
    normalized_default = "robinhood"

Robust input handling: Safely handles None, invalid values, and provides sensible fallback.

broker_filter.py:68-71 - Enhanced Idempotency

if getattr(mcp_server, _WRAPPER_ATTR, False):
    setattr(mcp_server, _ENABLED_ATTR, list(enabled_brokers))
    setattr(mcp_server, _DEFAULT_ATTR, normalized_default)
    return

Smart approach: Now updates both enabled brokers AND default broker on re-installation, ensuring full runtime reconfiguration.

broker_filter.py:78-98 - Runtime Configuration Access

async def _filtered_call_tool(tool_name: str, arguments: dict[str, Any]) -> Any:
    active_enabled: list[str] = getattr(mcp_server, _ENABLED_ATTR, ["robinhood"])
    active_default: str = getattr(mcp_server, _DEFAULT_ATTR, "robinhood")
    required = _tool_broker(tool_name, active_default)

Proper runtime lookup: Correctly retrieves current configuration on each call, enabling dynamic reconfiguration.

app.py:2304-2308 - Integration Point

install_broker_filter(
    server,
    config.brokers.enabled_brokers,
    default_broker=config.brokers.default_broker,
)

Perfect placement: Called after setup_brokers() but before transport starts, with proper DEFAULT_BROKER wiring.

🛡️ Security Assessment

  • No security vulnerabilities identified
  • Input validation: Environment variables properly parsed and validated
  • Error handling: No sensitive information leaked in error messages
  • Access control: Proper enforcement of broker-level access restrictions

⚡ Performance Considerations

  • Minimal overhead: Simple string matching and attribute lookups
  • No wrapper stacking: Idempotent design prevents performance degradation
  • Efficient filtering: frozenset lookup for agnostic tools is O(1)
  • Lazy evaluation: No unnecessary work when brokers are enabled

📝 Minor Suggestions

1. Documentation Enhancement (Optional)

Consider adding a brief docstring example to _tool_broker():

def _tool_broker(tool_name: str, default_broker: str) -> str | None:
    """Return the broker name required by *tool_name*, or ``None`` if agnostic.
    
    Examples:
        >>> _tool_broker("schwab_quote", "robinhood")
        "schwab"
        >>> _tool_broker("portfolio", "robinhood") 
        "robinhood"
        >>> _tool_broker("portfolio", "schwab")
        "schwab"
        >>> _tool_broker("broker_status", "robinhood")
        None
    """

2. Future-Proofing (Optional)

The substring matching for Schwab ("schwab" in tool_name) works well now but consider if you want to be more explicit for future brokers. Current approach is fine given the existing naming conventions.

🧪 Test Quality Assessment

Excellent test suite with these highlights:

  • Journey markers: Proper @pytest.mark.journey_system usage
  • Async testing: Correct use of @pytest.mark.asyncio
  • DEFAULT_BROKER coverage: Comprehensive testing of new default broker logic in tests 145-164
  • Clear test structure: Well-organized classes by functionality
  • Good assertions: Tests both positive and negative cases thoroughly
  • Idempotency testing: Tests 255-263 verify proper default broker updates on re-installation

🚀 Integration Assessment

Perfect integration approach:

  • Timing: Filter installed after broker setup, before transport start
  • Configuration: Uses centralized config.brokers for both enabled_brokers and default_broker
  • Non-breaking: Existing functionality unchanged when all brokers enabled
  • Graceful degradation: Clear error messages when brokers disabled
  • Full requirement coverage: Addresses both ENABLED_BROKERS and DEFAULT_BROKER from issue Config: ENABLED_BROKERS and DEFAULT_BROKER env vars ignored at runtime #991

Final Recommendation

✅ APPROVE - This PR is ready for merge. The implementation successfully addresses all requirements from issue #991, including the previously missing DEFAULT_BROKER enforcement. The code quality is high, test coverage is comprehensive (21 passing tests), and the integration is seamless. The latest changes properly honor the configured default broker for tool routing, making this a complete solution.


Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Config: ENABLED_BROKERS and DEFAULT_BROKER env vars ignored at runtime

2 participants