[AI] Server Prompt Templates #15402

paulb777 · 2025-10-09T22:48:38Z

Apple implementation of Server Prompt Templates inspired by Flutterfire PR.
Googlers: see API review at go/firebase-ai-server-prompt-template

Summary of Changes

This pull request significantly enhances the Firebase AI SDK by introducing support for server-side prompt templates. This allows developers to define and use reusable templates for generative AI models, making it easier to manage complex prompts and integrate dynamic data. The changes include new model types for generative and image AI, a dedicated chat session for template-based conversations, and a refactored, thread-safe history management system. These additions provide a more robust and flexible framework for building AI-powered applications.

Highlights

Server-Side Prompt Templates: Introduced new TemplateGenerativeModel and TemplateImagenModel classes to support server-side prompt templates, allowing for more flexible and dynamic AI interactions.
Template Chat Sessions: Added TemplateChatSession to enable conversational interactions with generative models using predefined templates and dynamic variables, supporting both single messages and streaming responses.
Refactored Chat History Management: Extracted chat history logic into a new, dedicated History class, improving modularity, reusability, and thread safety for managing conversation history in both standard and template-based chat sessions.
Dynamic Template Variables: Implemented TemplateVariable enum to handle various data types (String, Int, Double, Bool, Array, Dictionary) for substituting values into prompt templates, including automatic conversion for Float to Double.
New API Request Structures: Created specific request types (TemplateGenerateContentRequest, TemplateGenerateImagesRequest) and internal API method enums (APIMethod, ImageAPIMethod) to support the new template-based API endpoints.
Comprehensive Testing: Added extensive integration and unit tests for the new template models and chat sessions, covering various scenarios including text generation, image generation, and chat history management.

Changelog

FirebaseAI/Sources/Chat.swift
- Refactored to use the new History class for managing chat history, removing internal history management logic and related locks.
- Updated appendHistory calls to use the _history instance of the new History class.
FirebaseAI/Sources/FirebaseAI.swift
- Added public factory methods templateGenerativeModel() and templateImagenModel() to initialize new template-based AI models.
FirebaseAI/Sources/GenerateContentRequest.swift
- Removed the APIMethod enum, which has been moved to a new, dedicated internal file.
FirebaseAI/Sources/GenerativeAIService.swift
- Changed the firebaseInfo property from private let to let to allow access from newly introduced template models.
FirebaseAI/Sources/History.swift
- Added new file: Introduces a History class to encapsulate chat history management, including thread-safe appending of ModelContent and aggregation of content chunks.
FirebaseAI/Sources/TemplateChatSession.swift
- Added new file: Defines TemplateChatSession for managing chat conversations with server-side prompt templates, providing sendMessage and sendMessageStream methods.
FirebaseAI/Sources/TemplateGenerateContentRequest.swift
- Added new file: Defines the request structure for generating content using prompt templates, including template, variables, history, and project ID.
FirebaseAI/Sources/TemplateGenerateImagesRequest.swift
- Added new file: Defines the request structure for generating images using prompt templates, including template, variables, and project ID.
FirebaseAI/Sources/TemplateGenerativeModel.swift
- Added new file: Introduces TemplateGenerativeModel for interacting with generative AI models using templates, including methods for content generation and starting chat sessions.
FirebaseAI/Sources/TemplateImagenModel.swift
- Added new file: Introduces TemplateImagenModel for interacting with image generation models using templates.
FirebaseAI/Sources/TemplateVariable.swift
- Added new file: Defines TemplateVariable enum to handle various data types for template variables, including a conversion from Float to Double during initialization.
FirebaseAI/Sources/Types/Internal/APIMethod.swift
- Added new file: Moved the APIMethod enum (generateContent, streamGenerateContent, countTokens) here from GenerateContentRequest.swift.
FirebaseAI/Sources/Types/Internal/ImageAPIMethod.swift
- Added new file: Defines ImageAPIMethod enum specifically for image generation API methods, such as templatePredict.
FirebaseAI/Tests/TestApp/Tests/Integration/ServerPromptTemplateIntegrationTests.swift
- Added new file: Contains integration tests for TemplateGenerativeModel and TemplateImagenModel, covering text generation, image generation, and chat sessions with templates and media.
FirebaseAI/Tests/Unit/TemplateChatSessionTests.swift
- Added new file: Provides unit tests for the TemplateChatSession class, verifying sendMessage and sendMessageStream functionality.
FirebaseAI/Tests/Unit/TemplateGenerativeModelTests.swift
- Added new file: Includes unit tests for the TemplateGenerativeModel class, testing generateContent and generateContentStream methods.
FirebaseAI/Tests/Unit/TemplateImagenModelTests.swift
- Added new file: Contains unit tests for the TemplateImagenModel class, specifically testing the generateImages method.
FirebaseAI/Tests/Unit/TestUtilities/GenerativeModelTestUtil.swift
- Modified httpRequestHandler to include an isTemplateRequest parameter, allowing for conditional URL path assertions to support template-based requests in mock tests.
- Added a new helper function collectTextFromStream to simplify collecting text from AsyncThrowingStream responses in tests.

Activity

The Danger bot initially flagged a missing changelog entry.
A review comment from andrewheard suggested extracting chat history management into a separate type, which was subsequently implemented by paulb777.
Multiple automated review comments from gemini-code-assist[bot] highlighted potential issues with force unwrapping URLs in new request types and suggested percent-encoding template names for robustness. paulb777 acknowledged these, noting consistency with existing code.
A suggestion from gemini-code-assist[bot] to handle Float values in TemplateVariable initialization was addressed and implemented in the changes.
Other low-priority comments from gemini-code-assist[bot] included a TODO for streaming and a hardcoded path in a test utility.

google-oss-bot · 2025-10-09T22:49:31Z

	1 Warning
⚠️	Did you forget to add a changelog entry? (Add #no-changelog to the PR description to silence this warning.)

Generated by 🚫 Danger

andrewheard · 2025-10-09T22:56:57Z

FirebaseAI/Sources/TemplateChatSession.swift

+
+/// A chat session that allows for conversation with a model.
+@available(iOS 15.0, macOS 12.0, macCatalyst 15.0, tvOS 15.0, watchOS 8.0, *)
+public class TemplateChatSession {


Just a quick thing I noticed: public final class TemplateChatSession: Sendable. This would likely require some refactoring with the history. I'm wondering if we should extract

firebase-ios-sdk/FirebaseAI/Sources/Chat.swift

Lines 29 to 52 in 455d291

private let historyLock = NSLock()

private nonisolated(unsafe) var _history: [ModelContent] = []

/// The previous content from the chat that has been successfully sent and received from the

/// model. This will be provided to the model for each message sent as context for the discussion.

public var history: [ModelContent] {

get {

historyLock.withLock { _history }

}

set {

historyLock.withLock { _history = newValue }

}

}

private func appendHistory(contentsOf: [ModelContent]) {

historyLock.withLock {

_history.append(contentsOf: contentsOf)

}

}

private func appendHistory(_ newElement: ModelContent) {

historyLock.withLock {

_history.append(newElement)

}

}

into a type and use it for both?

Done in 004fb94

paulb777 · 2025-10-13T22:43:59Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces server-side prompt templates, a significant feature addition. The changes include new models (TemplateGenerativeModel, TemplateImagenModel), request types, and chat session logic to support using templates with variables. The code is well-structured, with good separation of concerns, such as moving history management into its own History class. I've identified a couple of areas where force unwraps on URL creation could be made safer. Overall, this is a solid implementation with good test coverage.

FirebaseAI/Sources/TemplateGenerateImagesRequest.swift

gemini-code-assist · 2025-10-13T22:51:06Z

FirebaseAI/Sources/TemplateGenerateContentRequest.swift

+      // TODO: Fix this.
+      urlString += "?alt=sse"
+    }
+    return URL(string: urlString)!


Using a force unwrap ! for URL creation can lead to a runtime crash if the URL string is malformed. It's safer to use guard let to safely unwrap the optional URL and provide a descriptive error message if it fails. This improves code robustness and debuggability.

guard let url = URL(string: urlString) else { fatalError("Could not construct URL for TemplateGenerateContentRequest. URL string was: \(urlString)") } return url

I think we should take this as an action item for the other request types too since we're force unwrapping in those. My first reaction was that it was unnecessary but given that location and template (model for the other requests) are entered by the dev it's definitely possible that the URL construction could fail.

FirebaseAI/Sources/TemplateGenerateContentRequest.swift

paulb777 · 2025-10-13T23:38:48Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces support for server-side prompt templates for both generative and image generation models. This is a significant feature addition, including new model types like TemplateGenerativeModel and TemplateImagenModel, new request structures, and a TemplateChatSession for conversational AI with templates. The changes also involve some nice refactoring, such as abstracting history management into a dedicated History class. The overall implementation is well-structured and comes with a good set of unit and integration tests. My feedback is focused on improving robustness by avoiding force-unwraps in URL construction, ensuring proper URL encoding of user-provided data, and making the TemplateVariable type more versatile by handling Float values. I've also included a minor suggestion to clean up test code.

FirebaseAI/Sources/GenerateImagesRequest.swift

FirebaseAI/Sources/TemplateGenerateContentRequest.swift

FirebaseAI/Sources/TemplateVariable.swift

FirebaseAI/Tests/TestApp/Tests/Integration/ServerPromptTemplateIntegrationTests.swift

FirebaseAI/Tests/Unit/TestUtilities/GenerativeModelTestUtil.swift

paulb777 · 2025-10-14T23:07:14Z

/gemini summarize

paulb777 · 2025-10-14T23:15:26Z

@andrewheard @cynthiajoan Still some test tweaks once the backend supports both API providers with the same templates and this won't merge until after M172, but should now be ready for review.

Updated `GenerateImagesRequest` to encode the `variables` property as `inputs`, similar to how `TemplateGenerateContentRequest` handles it. This ensures consistency across the API.

Updated `GenerateImagesRequest` to correctly construct the URL, including the `projectID` and `location`. This mirrors the URL construction in `TemplateGenerateContentRequest` and fixes a 404 error that was occurring in the `testGenerateImages` integration test. The `TemplateImagenModel` was also updated to pass the `projectID` to the `GenerateImagesRequest` initializer.

…tion supported

andrewheard · 2025-10-23T22:02:46Z

FirebaseAI/Sources/GenerateContentRequest.swift

-@available(iOS 15.0, macOS 12.0, macCatalyst 15.0, tvOS 15.0, watchOS 8.0, *)
-extension GenerateContentRequest {
-  enum APIMethod: String {
-    case generateContent
-    case streamGenerateContent
-    case countTokens
-  }
-}


I think we should consider undoing this (along with the addition of the file APIMethod.swift file since these custom verbs are specific to GenerateContentRequest.

andrewheard · 2025-10-23T22:05:14Z

FirebaseAI/Sources/TemplateGenerateContentRequest.swift

+    if case let .vertexAI(_, location) = apiConfig.service {
+      urlString += "/locations/\(location)"
+    }
+    urlString += "/templates/\(template):templateGenerateContent"


This should be templateStreamGenerateContent when stream == true.

Good catch! Done

andrewheard · 2025-10-23T22:05:49Z

FirebaseAI/Sources/Types/Internal/APIMethod.swift

Please undo.

andrewheard · 2025-10-23T22:11:03Z

FirebaseAI/Sources/TemplateGenerateImagesRequest.swift

+
+@available(iOS 15.0, macOS 12.0, macCatalyst 15.0, tvOS 15.0, watchOS 8.0, *)
+class TemplateGenerateImagesRequest: @unchecked Sendable, GenerativeAIRequest {


Could we follow the same pattern as ImagenGenerationRequest? That way it'll be prepared for images saved to GCS in the future.

Does this need to be a class? I think it could be a struct like ImagenGenerationRequest.

Any reason it needs to be @unchecked Sendable?

andrewheard · 2025-10-23T22:12:35Z

FirebaseAI/Sources/Types/Internal/ImageAPIMethod.swift

+enum ImageAPIMethod: String {
+  case generateImages = "templatePredict"
+}


I think this should be nested inside TemplateGenerateImagesRequest since it's very specific to SPT.

paulb777 · 2025-10-24T21:39:05Z

/gemini review

gemini-code-assist

Code Review

This pull request introduces a significant enhancement to the Firebase AI SDK by adding support for server-side prompt templates. The changes are well-structured, with new classes for template-based models (TemplateGenerativeModel, TemplateImagenModel) and a dedicated TemplateChatSession. The refactoring of chat history into a separate History class is a great improvement for modularity and thread safety. Additionally, the move from force-unwrapping URLs to a throwing mechanism enhances the robustness of the networking layer. My review focuses on ensuring URL components are properly encoded, maintaining consistency in error handling, and improving the maintainability of test utilities. Overall, this is a high-quality contribution.

FirebaseAI/Sources/TemplateGenerateContentRequest.swift

FirebaseAI/Sources/TemplateImagenGenerationRequest.swift

FirebaseAI/Sources/AILog.swift

FirebaseAI/Tests/Unit/TestUtilities/GenerativeModelTestUtil.swift

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

google-oss-bot added the api: firebaseai label Oct 9, 2025

andrewheard reviewed Oct 9, 2025

View reviewed changes

gemini-code-assist bot reviewed Oct 13, 2025

View reviewed changes

firebase deleted a comment from gemini-code-assist bot Oct 14, 2025

paulb777 marked this pull request as ready for review October 14, 2025 23:13

paulb777 added this to the 12.6.0 - M173 milestone Oct 14, 2025

paulb777 requested review from andrewheard and cynthiajoan October 14, 2025 23:16

paulb777 and others added 16 commits October 16, 2025 15:27

[AI] Server Prompt Templates

216a06d

copyrights

405d5fe

Checkpoint after trying to add imagen implementation

c3c03be

end to end testGenerateContentWithText passes

953b3f0

FirebaseAI/Tests/Unit/TemplateGenerativeModelTests.swift

daf7b4e

CI fixes

29c79b1

Streaming APIs

6aa13e0

chat history refactor

e75d7e9

Re-sort tests

6cc6c9d

Fix TemplateImagenModel to encode variables as inputs

ac996fd

Updated `GenerateImagesRequest` to encode the `variables` property as `inputs`, similar to how `TemplateGenerateContentRequest` handles it. This ensures consistency across the API.

checkpoint

dd66187

Imagen generate test works

b3c28d1

fix template imagen response return type

705d0c2

existing tests passing

17fd0b8

build fix

c4068e9

paulb777 added 12 commits October 16, 2025 15:27

style

7af117c

Add streaming integration tests

32bc581

testChatStream

1d162dc

unit tests run

d7beef0

streaming unit tests

fd8f76a

existing mock files for template unit tests

a49dee1

streamline integration tests

3f49b09

fixes

3f3607a

Templates do not have an implicit .prompt suffix and only global loca…

64df672

…tion supported

fixes

9f91c21

self review, including file/naming updates

fa4a3f4

Update Integration test xcodeproj after rebase

7bd5d2c

paulb777 force-pushed the pb-spt branch from 9b1d73c to 7bd5d2c Compare October 16, 2025 22:36

paulb777 and others added 8 commits October 16, 2025 16:12

module rename updates after rebase for unit tests

eeb9e66

[Firebase AI] Refactor SPT integration tests to Swift Testing (#15426)

1840566

Continue migration from template templates to project templates

67d40eb

Rename template variables to template inputs

6338983

template to templateID

a11c20c

doc updates

bc4ab28

Add sendMessage modelContent variations

5c784c7

Convert tests to use test project prompt templates

60f32ef

andrewheard reviewed Oct 23, 2025

View reviewed changes

paulb777 added 3 commits October 24, 2025 13:55

Eliminate force unwraps in url creation

fe3ae25

misc review fixes

f63c4c0

aligned structure and naming

83a0f3a

gemini-code-assist bot reviewed Oct 24, 2025

View reviewed changes

paulb777 and others added 2 commits October 24, 2025 14:44

Update FirebaseAI/Sources/AILog.swift

9ee5acc

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Build fixes

287dd12

	private let historyLock = NSLock()
	private nonisolated(unsafe) var _history: [ModelContent] = []
	/// The previous content from the chat that has been successfully sent and received from the
	/// model. This will be provided to the model for each message sent as context for the discussion.
	public var history: [ModelContent] {
	get {
	historyLock.withLock { _history }
	}
	set {
	historyLock.withLock { _history = newValue }
	}
	}

	private func appendHistory(contentsOf: [ModelContent]) {
	historyLock.withLock {
	_history.append(contentsOf: contentsOf)
	}
	}

	private func appendHistory(_ newElement: ModelContent) {
	historyLock.withLock {
	_history.append(newElement)
	}
	}


		@available(iOS 15.0, macOS 12.0, macCatalyst 15.0, tvOS 15.0, watchOS 8.0, *)
		class TemplateGenerateImagesRequest: @unchecked Sendable, GenerativeAIRequest {

[AI] Server Prompt Templates #15402

Are you sure you want to change the base?

[AI] Server Prompt Templates #15402

Uh oh!

Conversation

paulb777 commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of Changes

Highlights

Uh oh!

google-oss-bot commented Oct 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulb777 commented Oct 13, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

gemini-code-assist bot Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

paulb777 commented Oct 13, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

paulb777 commented Oct 14, 2025

Uh oh!

paulb777 commented Oct 14, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

paulb777 commented Oct 24, 2025

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

paulb777 commented Oct 9, 2025 •

edited

Loading