Skip to content

LLMInference Should Delegate to Functional API #7

@eric-tramel

Description

@eric-tramel

Description

The LLMInference class (in module.py) should delegate to or be implemented in terms of the functional API (in functional.py) rather than having a separate, parallel implementation.

Currently there may be duplication between:

  • LLMInference.forward() - the module-based approach
  • Functions in functional.py - the stateless, Value-aware approach

Having two separate implementations risks:

  1. Inconsistent behavior between module and functional styles
  2. Duplicated bug fixes and improvements
  3. Confusion about which approach is canonical
  4. Divergent handling of Value/ValueRef semantics

The LLMInference class should be a thin wrapper that internally uses the functional API, ensuring a single source of truth for LLM call behavior.

Acceptance Criteria

  • Audit LLMInference implementation in module.py
  • Identify functional API functions that handle LLM calls (or create them if missing)
  • Refactor LLMInference.forward() to delegate to functional API
  • Ensure Value/ValueRef handling is consistent between both interfaces
  • Maintain backward compatibility with existing LLMInference usage
  • Update tests to verify delegation works correctly
  • Update documentation to clarify relationship between module and functional styles

Notes

  • The functional API in functional.py operates on Value objects and propagates errors as values
  • LLMInference is defined in module.py as a subclass of InferenceModule
  • This may require adding LLM-specific functions to functional.py if they don't exist

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions