Skip to content

Enable mixing regular and built-in tools in Gemini 3+#1055

Draft
thisisnic wants to merge 6 commits into
tidyverse:mainfrom
thisisnic:1054-mixtools
Draft

Enable mixing regular and built-in tools in Gemini 3+#1055
thisisnic wants to merge 6 commits into
tidyverse:mainfrom
thisisnic:1054-mixtools

Conversation

@thisisnic

Copy link
Copy Markdown
Collaborator

Fixes #1054

The ability to combine built-in and existing tools was an addition in Gemini 3.

Previously, combining built-in and custom tools resulted in an error; this PR sets parameter includeServerSideToolInvocations to TRUE, which allows them to be combined.

Didn't add a test as it feels like more of a minor JSON configuration update than a true bug fix, but before/after are here.

Before:

library(ellmer)
chat <- chat_google_gemini()
#> Using model = "gemini-3.5-flash".
chat$register_tool(tool(
  function(x) x * 2,
  "Doubles a number",
  arguments = list(x = type_number("The number to double"))
))
chat$register_tool(google_tool_web_search())
chat$chat("What is double today's temperature in the UK?")
#> Error in `req_perform_connection()` at ellmer/R/httr2.R:55:5:
#> ! HTTP 400 Bad Request.
#> ℹ Please enable tool_config.include_server_side_tool_invocations to use
#>   Built-in tools with Function calling.

After:

library(ellmer)
chat <- chat_google_gemini()
#> Using model = "gemini-3.5-flash".
chat$register_tool(tool(
  function(x) x * 2,
  "Doubles a number",
  arguments = list(x = type_number("The number to double"))
))
chat$register_tool(google_tool_web_search())
chat$chat("What is double today's temperature in the UK?")
#> ◯ [tool call] tool_001(x = 21L)
#> ● #> 42
#> ◯ [tool call] tool_001(x = 33L)
#> ● #> 66
#> ◯ [tool call] tool_001(x = 34L)
#> ● #> 68
#> ◯ [tool call] tool_001(x = 23L)
#> ● #> 46
#> ◯ [tool call] tool_001(x = 70L)
#> ● #> 140
#> ◯ [tool call] tool_001(x = 94L)
#> ● #> 188
#> Currently in London, UK (as of ~7:00 AM BST), the temperature is approximately 
#> **21°C to 23°C** (around **70°F**), with a forecast high today reaching up to 
#> **33°C to 34°C** (around **91°F to 94°F**) [1, 2]. 
#> 
#> Depending on whether you are looking at the current temperature or today's 
#> forecast high (in Celsius or Fahrenheit), doubling the temperature yields the 
#> following:
#> 
#> ### 1. Doubling the Current Temperature (~21°C to 23°C / 70°F)
#> * **In Celsius:** Doubling 21°C to 23°C gives **42°C to 46°C**.
#> * **In Fahrenheit:** Doubling 70°F gives **140°F**.
#> 
#> ### 2. Doubling Today's Forecast High (~33°C to 34°C / 91°F to 94°F)
#> * **In Celsius:** Doubling 33°C to 34°C gives **66°C to 68°C**.
#> * **In Fahrenheit:** Doubling 91°F to 94°F gives **182°F to 188°F**.

@thisisnic thisisnic marked this pull request as ready for review July 9, 2026 07:08
@thisisnic thisisnic marked this pull request as draft July 9, 2026 07:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Google Gemini errors when mixing regular and builtin tools

1 participant