[WIP] Model Conversion Framework for EAGLE and HASS

[rahul-tuli]

Model Conversion Framework for EAGLE and HASS

Summary

This PR introduces a model conversion framework that enables users to convert existing EAGLE (v1/v3) and HASS speculative decoding models into the standardized Speculators format. The framework provides both a programmatic API and CLI interface for seamless conversion workflows.

Key Features:

• Unified converter for EAGLE variants (v1, v2) and HASS models
• Extensible base converter class for future algorithms
• CLI command: speculators convert for easy command-line usage

Implementation Details

Base Converter (converters/base.py)

class BaseConverter(ABC):
    def convert(source_path, output_path, verifier_model, ...) -> ConversionResult
    def load_source_model(path) -> Tuple[config, weights]
    def load_verifier_config(verifier_model) -> Dict
    def save_converted_model(output_path, config, weights) -> None
    
    @abstractmethod
    def validate_source(config) -> List[str]
    def map_config(source_config, verifier_config, ...) -> Dict
    def map_weights(source_weights, source_config) -> Tuple[Dict, List]

Configuration Mapping

1. Detect variant from source config (model_type, architectures)
2. Create LlamaEagleSpeculatorConfig with appropriate settings
3. Add speculators_config with algorithm info and verifier details
4. Include default proposal methods based on variant

Usage Examples

Programmatic API

from speculators import convert

# Convert EAGLE model
result = convert(
    source_path="/path/to/eagle/model",
    output_path="/path/to/output",
    verifier_model="meta-llama/Llama-3.1-8B-Instruct",
    algorithm="eagle"
)

if result.success:
    print("Conversion successful!")
    if result.warnings:
        print(f"Warnings: {result.warnings}")
else:
    print(f"Conversion failed: {result.errors}")

CLI

# Basic conversion
speculators convert /path/to/eagle/model /path/to/output \
    --verifier meta-llama/Llama-3.1-8B-Instruct \
    --algorithm eagle

# With custom proposal methods
speculators convert /path/to/hass/model /path/to/output \
    --verifier meta-llama/Llama-3.1-70B \
    --algorithm hass \
    --proposal-methods '[{"proposal_type": "greedy", "draft_tokens": 10}]'

# Force overwrite existing output
speculators convert /path/to/model /path/to/output \
    --verifier meta-llama/Llama-3.1-8B \
    --algorithm eagle \
    --force

Files Added/Modified

New Files

• src/speculators/converters/__init__.py - Package exports
• src/speculators/converters/base.py - Base converter class and utilities
• src/speculators/converters/llama_eagle.py - EAGLE/HASS converter implementation
• src/speculators/convert.py - High-level conversion API
• src/speculators/cli/__init__.py - CLI package exports
• src/speculators/cli/main.py - Main CLI entry point
• src/speculators/cli/convert.py - Convert command implementation

Modified Files

• src/speculators/__init__.py - Added convert to exports
• pyproject.toml - Added typer dependency and CLI entry point

Checklist

• Conversion framework implemented
• EAGLE/HASS converter working
• High-level API exposed
• CLI command implementation
• Unit tests
• Integration tests
• Documentation updates

[nm-red-hat-upstream-automation-bot]

📦 Build Artifacts Available
The build artifacts (.whl and .tar.gz) have been successfully generated and are available for download: https://github.com/neuralmagic/speculators/actions/runs/15617818019/artifacts/3316866369.
They will be retained for up to 30 days.