Skip to content

fix(firebase_ai): parse promptTokensDetails and empty CountTokensResponse on GoogleAI - #18916

Open
cynthiajoan wants to merge 2 commits into
firebase:mainfrom
cynthiajoan:agent/count-tokens-tts-firebase_ai
Open

cynthiajoan wants to merge 2 commits into
firebase:mainfrom
cynthiajoan:agent/count-tokens-tts-firebase_ai

Conversation

@cynthiajoan

Copy link
Copy Markdown
Contributor

Description

DeveloperSerialization.parseCountTokensResponse (lib/src/developer/api.dart) dropped promptTokensDetails returned by the Gemini Developer API (googleAI backend), so per-modality token counts such as TEXT and AUDIO were always null, and it threw Unhandled format for CountTokensResponse: {} when the server omitted totalTokens (protobuf JSON omits zero values).

parseCountTokensResponse now returns CountTokensResponse(0) for an empty object and parses promptTokensDetails with the existing parseUsageMetadata helper, matching the agentPlatform serializer. The LiveSession.receive() doc comment is also corrected: the stream stays open across turns until the session is closed.

Verification

  • Change kind: bugfix
  • Red (base d07113e3a + only the new tests): flutter test --no-pub test/developer_api_test.dart → exit 1, 25 passed, 2 failed; new tests failing: parses promptTokensDetails when present, defaults totalTokens to 0 on empty JSON object:
    00:00 +20 -1: DeveloperSerialization parseCountTokensResponse parses promptTokensDetails when present [E]
      Expected: not null
        Actual: <null>
      package:matcher                                     expect
      package:flutter_test/src/widget_tester.dart 473:18  expect
      test/developer_api_test.dart 636:9                  main.<fn>.<fn>.<fn>
    00:00 +20 -1: DeveloperSerialization parseCountTokensResponse defaults totalTokens to 0 on empty JSON object
    00:00 +20 -2: DeveloperSerialization parseCountTokensResponse defaults totalTokens to 0 on empty JSON object [E]
      Unhandled format for CountTokensResponse: {}
      This indicates a problem with the Firebase AI Logic SDK. Try updating to the latest version (https://pub.dev/packages/firebase_ai/versions), or file an issue at https://github.com/firebase/flutterfire/issues.
      package:firebase_ai/src/developer/api.dart 106:5  DeveloperSerialization.parseCountTokensResponse
      test/developer_api_test.dart 645:51               main.<fn>.<fn>.<fn>
    00:00 +20 -2: DeveloperSerialization parseCountTokensResponse throws FirebaseAIException on error response
    00:00 +21 -2: DeveloperSerialization parseCountTokensResponse throws unhandledFormat on invalid JSON
    00:00 +22 -2: DeveloperSerialization generateContentRequest serializes safetySettings correctly for developer API
    00:00 +23 -2: DeveloperSerialization generateContentRequest throws ArgumentError for safetySetting with method
    00:00 +24 -2: DeveloperSerialization countTokensRequest serializes request with generateContentRequest wrapper
    00:00 +25 -2: Some tests failed.
    
  • Green (head 3ca476f21): flutter test --no-pub test/developer_api_test.dart → exit 0, 27 passed (4s).
  • Fork CI: fork CI green: all_plugins, e2e-ai, e2e-smoke, pr_title

Related Issues

N/A (AI-COUNT-TOKENS-1)

Checklist

Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]).
This will ensure a smooth and quick review process. Updating the pubspec.yaml and changelogs is not required.

  • I read the Contributor Guide and followed the process outlined there for submitting PRs.
  • My PR includes unit or integration tests for all changed/updated/fixed behaviors (See Contributor Guide).
  • All existing and new tests are passing.
  • I updated/added relevant documentation (doc comments with ///).
  • The analyzer (melos run analyze) does not report any problems on my PR.
  • I read and followed the Flutter Style Guide.
  • I signed the CLA.
  • I am willing to follow-up on review comments in a timely manner.

Breaking Change

Does your PR require plugin users to manually update their apps to accommodate your change?

  • Yes, this is a breaking change.
  • No, this is not a breaking change.

Prepared with the Firebase SDK dev-loop agent (Google-internal, built on Jetski) and reviewed by @cynthiajoan before opening. Evidence: red/green captured by verify-redgreen (verdict verified) at base d07113e3a / head 3ca476f21.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates parseCountTokensResponse in the developer API to parse promptTokensDetails and handle empty JSON objects, updates documentation for LiveSession.receive(), and adds corresponding unit tests. Feedback highlights a potential parsing failure if totalTokens is omitted while promptTokensDetails is present, and recommends avoiding the ! operator in tests to comply with the repository's null safety style guide.

Comment on lines 109 to 121
if (jsonObject case {'totalTokens': final int totalTokens}) {
return CountTokensResponse(totalTokens);
final promptTokensDetails = switch (jsonObject) {
{'promptTokensDetails': final List<Object?> details} =>
parseUsageMetadata({
'promptTokensDetails': details,
}).promptTokensDetails,
_ => null,
};
return CountTokensResponse(
totalTokens,
promptTokensDetails: promptTokensDetails,
);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

If totalTokens is omitted (e.g., because it is 0 and omitted in protobuf JSON) but promptTokensDetails is present, the pattern match jsonObject case {'totalTokens': final int totalTokens} will fail, causing the method to throw an unhandledFormat exception. We should make the parsing more robust by checking if either totalTokens or promptTokensDetails is present, and defaulting totalTokens to 0 if it is missing.

    final hasTotalTokens = jsonObject.containsKey('totalTokens');
    final hasPromptTokensDetails = jsonObject.containsKey('promptTokensDetails');
    if (hasTotalTokens || hasPromptTokensDetails) {
      final totalTokens = jsonObject['totalTokens'] as int? ?? 0;
      final promptTokensDetails = switch (jsonObject) {
        {'promptTokensDetails': final List<Object?> details} =>
          parseUsageMetadata({
            'promptTokensDetails': details,
          }).promptTokensDetails,
        _ => null,
      };
      return CountTokensResponse(
        totalTokens,
        promptTokensDetails: promptTokensDetails,
      );
    }

Comment on lines +636 to +641
expect(details, isNotNull);
expect(details, hasLength(2));
expect(details![0].modality, ContentModality.text);
expect(details[0].tokenCount, 100);
expect(details[1].modality, ContentModality.audio);
expect(details[1].tokenCount, 23);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

low

Avoid using the ! operator as per the repository style guide (Rule 23). Instead, use flow analysis with a null check (if (details != null)) to safely access the list elements.

        expect(details, isNotNull);
        if (details != null) {
          expect(details, hasLength(2));
          expect(details[0].modality, ContentModality.text);
          expect(details[0].tokenCount, 100);
          expect(details[1].modality, ContentModality.audio);
          expect(details[1].tokenCount, 23);
        }
References
  1. Null Safety: NO ! operator. Use ? and flow analysis (e.g. if (x != null)). (link)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant