AnyLanguageModel

A Swift package that provides a drop-in replacement for Apple's Foundation Models framework with support for custom language model providers. All you need to do is change your import statement:

- import FoundationModels
+ import AnyLanguageModel

struct WeatherTool: Tool {
    let name = "getWeather"
    let description = "Retrieve the latest weather information for a city"

    @Generable
    struct Arguments {
        @Guide(description: "The city to fetch the weather for")
        var city: String
    }

    func call(arguments: Arguments) async throws -> String {
        "The weather in \(arguments.city) is sunny and 72°F / 23°C"
    }
}

let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model, tools: [WeatherTool()])

let response = try await session.respond {
    Prompt("How's the weather in Cupertino?")
}
print(response.content)

Features

Supported Providers

• Apple Foundation Models
• Core ML models
• MLX models
• llama.cpp (GGUF models)
• Ollama HTTP API
• Anthropic Messages API
• Google Gemini API
• OpenAI Chat Completions API
• OpenAI Responses API

Requirements

• Swift 6.1+
• iOS 17.0+ / macOS 14.0+ / visionOS 1.0+ / Linux

Important

A bug in Xcode 26 may cause build errors when targeting macOS 15 / iOS 18 or earlier (e.g. Conformance of 'String' to 'Generable' is only available in macOS 26.0 or newer). As a workaround, build your project with Xcode 16. For more information, see issue #15.

Installation

Add this package to your Package.swift:

dependencies: [
    .package(url: "https://github.com/mattt/AnyLanguageModel", from: "0.4.0")
]

Package Traits

AnyLanguageModel uses Swift 6.1 traits to conditionally include heavy dependencies, allowing you to opt-in only to the language model backends you need. This results in smaller binary sizes and faster build times.

Available traits:

• CoreML: Enables Core ML model support (depends on huggingface/swift-transformers)
• MLX: Enables MLX model support (depends on ml-explore/mlx-swift-lm)
• Llama: Enables llama.cpp support (requires mattt/llama.swift)

By default, no traits are enabled. To enable specific traits, specify them in your package's dependencies:

// In your Package.swift
dependencies: [
    .package(
        url: "https://github.com/mattt/AnyLanguageModel.git",
        branch: "main",
        traits: ["CoreML", "MLX"] // Enable CoreML and MLX support
    )
]

Using Traits in Xcode Projects

Xcode doesn't yet provide a built-in way to declare package dependencies with traits. As a workaround, you can create an internal Swift package that acts as a shim, exporting the AnyLanguageModel module with the desired traits enabled. Your Xcode project can then add this internal package as a local dependency.

For example, to use AnyLanguageModel with MLX support in an Xcode app project:

1. Create a local Swift package (in root directory containing Xcode project):

mkdir -p Packages/MyAppKit
cd Packages/MyAppKit
swift package init

2. Specify AnyLanguageModel package dependency (in Packages/MyAppKit/Package.swift):

// swift-tools-version: 6.1
import PackageDescription

let package = Package(
    name: "MyAppKit",
    platforms: [
        .macOS(.v14),
        .iOS(.v17),
        .visionOS(.v1),
    ],
    products: [
        .library(
            name: "MyAppKit",
            targets: ["MyAppKit"]
        )
    ],
    dependencies: [
        .package(
            url: "https://github.com/mattt/AnyLanguageModel",
            from: "0.4.0",
            traits: ["MLX"]
        )
    ],
    targets: [
        .target(
            name: "MyAppKit",
            dependencies: [
                .product(name: "AnyLanguageModel", package: "AnyLanguageModel")
            ]
        )
    ]
)

3. Export the AnyLanguageModel module (in Sources/MyAppKit/Export.swift):

@_exported import AnyLanguageModel

4. Add the local package to your Xcode project:

Open your project settings, navigate to the "Package Dependencies" tab, and click "+" → "Add Local..." to select the Packages/MyAppKit directory.

Your app can now import AnyLanguageModel with MLX support enabled.

Tip

For a working example of package traits in an Xcode app project, see chat-ui-swift.

API Credentials and Security

When using third-party language model providers like OpenAI, Anthropic, or Google Gemini, you must handle API credentials securely.

Caution

Never hardcode API credentials in your app. Malicious actors can reverse‑engineer your application binary or observe outgoing network requests (for example, on a compromised device or via a debugging proxy) to extract embedded credentials. There have been documented cases of attackers successfully exfiltrating API keys from mobile apps and racking up thousands of dollars in charges.

Here are two approaches for managing API credentials in production apps:

Bring Your Own Key (BYO)

Users provide their own API keys, which are stored securely in the system Keychain and sent directly to the provider in API requests.

Security considerations:

• Keychain data is encrypted using hardware-backed keys (protected by the Secure Enclave on supported devices)
• An attacker would need access to a running process to intercept credentials
• TLS encryption protects credentials in transit on the network
• Users can only compromise their own keys, not other users' keys

Trade-offs:

• Apple App Review has often rejected apps using this model
• Reviewers may be unable to test functionality — even with provided credentials
• Apple may require in-app purchase integration for usage credits
• Some users may find it inconvenient to obtain and enter API keys

Proxy Server

Instead of connecting directly to the provider, route requests through your own authenticated service endpoint. API credentials are stored securely on your server, never in the client app.

Authenticate users with OAuth 2.1 or similar, issuing short-lived, scoped bearer tokens for client requests. If an attacker extracts tokens from your app, they're limited in scope and expire automatically.

Security considerations:

• API keys never leave your server infrastructure
• Client tokens can be scoped (e.g., rate-limited, feature-restricted)
• Client tokens can be revoked or expired independently
• Compromised tokens have limited blast radius

Trade-offs:

• Additional infrastructure complexity (server, authentication, monitoring)
• Operational costs (hosting, maintenance, support)
• Network latency from additional hop

Fortunately, there are platforms and services that simplify proxy implementation, handling authentication, rate limiting, and billing for you.

Tip

For development and testing, it's fine to use API keys from environment variables. Just make sure production builds use one of the secure approaches above.

For more information about security best practices for your app, see OWASP's Mobile Application Security Cheat Sheet.

Usage

Apple Foundation Models

Uses Apple's system language model (requires macOS 26 / iOS 26 / visionOS 26 or later).

let model = SystemLanguageModel.default
let session = LanguageModelSession(model: model)

let response = try await session.respond {
    Prompt("Explain quantum computing in one sentence")
}

Note

Image inputs are not yet supported by Apple Foundation Models.

Core ML

Run Core ML models (requires CoreML trait):

let model = CoreMLLanguageModel(url: URL(fileURLWithPath: "path/to/model.mlmodelc"))

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Summarize this text")
}

Enable the trait in Package.swift:

.package(
    url: "https://github.com/mattt/AnyLanguageModel.git",
    branch: "main",
    traits: ["CoreML"]
)

Note

Image inputs are not currently supported with CoreMLLanguageModel.

MLX

Run MLX models on Apple Silicon (requires MLX trait):

let model = MLXLanguageModel(modelId: "mlx-community/Qwen3-0.6B-4bit")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("What is the capital of France?")
}

Vision support depends on the specific MLX model you load. Use a vision‑capable model for multimodal prompts (for example, a VLM variant). The following shows extracting text from an image:

let ocr = try await session.respond(
    to: "Extract the total amount from this receipt",
    images: [
        .init(url: URL(fileURLWithPath: "/path/to/receipt_page1.png")),
        .init(url: URL(fileURLWithPath: "/path/to/receipt_page2.png"))
    ]
)
print(ocr.content)

Enable the trait in Package.swift:

.package(
    url: "https://github.com/mattt/AnyLanguageModel.git",
    branch: "main",
    traits: ["MLX"]
)

llama.cpp (GGUF)

Run GGUF quantized models via llama.cpp (requires Llama trait):

let model = LlamaLanguageModel(modelPath: "/path/to/model.gguf")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Translate 'hello world' to Spanish")
}

Enable the trait in Package.swift:

.package(
    url: "https://github.com/mattt/AnyLanguageModel.git",
    branch: "main",
    traits: ["Llama"]
)

Note

Image inputs are not currently supported with LlamaLanguageModel.

OpenAI

Supports both Chat Completions and Responses APIs:

let model = OpenAILanguageModel(
    apiKey: ProcessInfo.processInfo.environment["OPENAI_API_KEY"]!,
    model: "gpt-4o-mini"
)

let session = LanguageModelSession(model: model)
let response = try await session.respond(
    to: "List the objects you see",
    images: [
        .init(url: URL(string: "https://example.com/desk.jpg")!),
        .init(
            data: try Data(contentsOf: URL(fileURLWithPath: "/path/to/closeup.png")),
            mimeType: "image/png"
        )
    ]
)
print(response.content)

For OpenAI-compatible endpoints that use older Chat Completions API:

let model = OpenAILanguageModel(
    baseURL: URL(string: "https://api.example.com")!,
    apiKey: apiKey,
    model: "gpt-4o-mini",
    apiVariant: .chatCompletions
)

Anthropic

Uses the Messages API with Claude models:

let model = AnthropicLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]!,
    model: "claude-sonnet-4-5-20250929"
)

let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
    Prompt("What's the weather like in San Francisco?")
}

You can include images with your prompt. You can point to remote URLs or construct from image data:

let response = try await session.respond(
    to: "Explain the key parts of this diagram",
    image: .init(
        data: try Data(contentsOf: URL(fileURLWithPath: "/path/to/diagram.png")),
        mimeType: "image/png"
    )
)
print(response.content)

Google Gemini

Uses the Gemini API with Gemini models:

let model = GeminiLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["GEMINI_API_KEY"]!,
    model: "gemini-2.5-flash"
)

let session = LanguageModelSession(model: model, tools: [WeatherTool()])
let response = try await session.respond {
    Prompt("What's the weather like in Tokyo?")
}

Send images with your prompt using remote or local sources:

let response = try await session.respond(
    to: "Identify the plants in this photo",
    image: .init(url: URL(string: "https://example.com/garden.jpg")!)
)
print(response.content)

Gemini models use an internal "thinking process" that improves reasoning and multi-step planning. You can configure how much Gemini should "think" using the thinking parameter:

// Enable thinking
var model = GeminiLanguageModel(
    apiKey: apiKey,
    model: "gemini-2.5-flash",
    thinking: true /* or `.dynamic` */,
)

// Set an explicit number of tokens for its thinking budget
model.thinking = .budget(1024)

// Revert to default configuration without thinking
model.thinking = false /* or `.disabled` */

Gemini supports server-side tools that execute transparently on Google's infrastructure:

let model = GeminiLanguageModel(
    apiKey: apiKey,
    model: "gemini-2.5-flash",
    serverTools: [
        .googleMaps(latitude: 35.6580, longitude: 139.7016) // Optional location
    ]
)

Available server tools:

• .googleSearch Grounds responses with real-time web information
• .googleMaps Provides location-aware responses
• .codeExecution Generates and runs Python code to solve problems
• .urlContext Fetches and analyzes content from URLs mentioned in prompts

Tip

Gemini server tools are not available as client tools (Tool) for other models.

Ollama

Run models locally via Ollama's HTTP API:

// Default: connects to http://localhost:11434
let model = OllamaLanguageModel(model: "qwen3") // `ollama pull qwen3:8b`

// Custom endpoint
let model = OllamaLanguageModel(
    endpoint: URL(string: "http://remote-server:11434")!,
    model: "llama3.2"
)

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Tell me a joke")
}

For local models, make sure you’re using a vision‑capable model (for example, a -vl variant). You can combine multiple images:

let model = OllamaLanguageModel(model: "qwen3-vl") // `ollama pull qwen3-vl:8b`
let session = LanguageModelSession(model: model)
let response = try await session.respond(
    to: "Compare these posters and summarize their differences",
    images: [
        .init(url: URL(string: "https://example.com/poster1.jpg")!),
        .init(url: URL(fileURLWithPath: "/path/to/poster2.jpg"))
    ]
)
print(response.content)

Testing

Run the test suite to verify everything works correctly:

swift test

Tests for different language model backends have varying requirements:

• CoreML tests: swift test --traits CoreML + ENABLE_COREML_TESTS=1 + HF_TOKEN (downloads model from HuggingFace)
• MLX tests: swift test --traits MLX + ENABLE_MLX_TESTS=1 + HF_TOKEN (uses pre-defined model)
• Llama tests: swift test --traits Llama + LLAMA_MODEL_PATH (points to local GGUF file)
• Anthropic tests: ANTHROPIC_API_KEY (no traits needed)
• OpenAI tests: OPENAI_API_KEY (no traits needed)
• Ollama tests: No setup needed (skips in CI)

Example setup for all backends:

# Environment variables
export ENABLE_COREML_TESTS=1
export ENABLE_MLX_TESTS=1
export HF_TOKEN=your_huggingface_token
export LLAMA_MODEL_PATH=/path/to/model.gguf
export ANTHROPIC_API_KEY=your_anthropic_key
export OPENAI_API_KEY=your_openai_key

# Run all tests with traits enabled
swift test --traits CoreML,MLX,Llama

License

This project is available under the MIT license. See the LICENSE file for more info.