Skip to content

[FEATURE][PLUGINS]: Tool call anomaly detection plugin #3845

@anujshrivastava15

Description

@anujshrivastava15

🧭 Type of Feature

  • New feature or capability

🧭 Epic

Title: Tool Call Anomaly Detection Plugin
Goal: Detect unusual tool-calling patterns per user/agent by learning behavioral baselines.
Why now: ContextForge has strong permission-based controls (RBAC, PDP) and content-based security (PII filter, secrets detection), but no way to flag when a user's tool-calling behavior changes — e.g. sudden access to new tools, burst patterns, or off-hours activity. This would complement existing security and feed data into the planned Security Posture Dashboard (#2309).


🧑🏻‍💻 User Story 1

As a: platform administrator
I want: automatic detection when a user/agent's tool-calling pattern deviates from their baseline
So that: I'm alerted to potentially compromised credentials or prompt injection — even when RBAC permits the access.

✅ Acceptance Criteria

Scenario: Known tool call scores low risk
  Given user "alice" regularly calls "db_query"
  When "alice" calls "db_query" with typical arguments
  Then the plugin allows the call with risk_score < 0.5

Scenario: Novel tool triggers elevated risk
  Given user "alice" has never called "delete_records"
  When "alice" calls "delete_records"
  Then risk_score is elevated and a warning is logged

Scenario: Burst pattern detected
  Given burst_threshold is 20 calls per 60 seconds
  When a user makes 25 calls in 30 seconds
  Then burst score exceeds threshold

📐 Design Sketch

flowchart TD
    A[Tool Call] --> B{Learning phase?}
    B -->|Yes| C[Record baseline, allow]
    B -->|No| D[Score: novelty + burst + frequency + off-hours]
    D --> E{Above threshold?}
    E -->|Block mode| F[Block + violation]
    E -->|Warn mode| G[Allow + warn log]
    E -->|Below| H[Allow + metadata]
Loading
  • Hooks: tool_pre_invoke, tool_post_invoke
  • Dependencies: None — pure Python, in-memory baselines
  • Priority: Runs after RBAC/PDP checks

🔗 MCP Standards Check

  • Change adheres to current MCP specifications
  • No breaking changes to existing MCP-compliant integrations

🔄 Alternatives Considered

None at this stage.


📓 Additional Context

None.

Metadata

Metadata

Assignees

No one assigned

    Labels

    WOULDP4: Not a priority for current scope; very likely to move to future releasesenhancementNew feature or requestplugins

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions