Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion crates/tool_parser/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Parser library for extracting tool/function calls from LLM model outputs. Suppor
| `LlamaParser` | Llama 3.2 | `<\|python_tag\|>{...}` |
| `PythonicParser` | Llama 4, DeepSeek R1 | `[func_name(arg="val")]` |
| `DeepSeekParser` | DeepSeek V3 | `<\|tool▁calls▁begin\|>...<\|tool▁calls▁end\|>` |
| `Glm4MoeParser` | GLM-4.5/4.6/4.7 | `<\|observation\|>...<\|/observation\|>` |
| `GlmParser` | GLM-4.5 through GLM-5.1 | `<tool_call>...<arg_key>...</arg_key><arg_value>...</arg_value></tool_call>` |
| `Step3Parser` | Step-3 | `<steptml:function_call>...</steptml:function_call>` |
| `KimiK2Parser` | Kimi K2 | `<\|tool_call_begin\|>...<\|tool_call_end\|>` |
| `MinimaxM2Parser` | MiniMax M2 | `<FUNCTION_CALL>{...}</FUNCTION_CALL>` |
Expand Down
17 changes: 8 additions & 9 deletions crates/tool_parser/src/factory.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ use tokio::sync::Mutex;

use crate::{
parsers::{
CohereParser, DeepSeek31Parser, DeepSeekDsmlParser, DeepSeekParser, Glm4MoeParser,
JsonParser, KimiK2Parser, LlamaParser, MinimaxM2Parser, MistralParser, PassthroughParser,
CohereParser, DeepSeek31Parser, DeepSeekDsmlParser, DeepSeekParser, GlmParser, JsonParser,
KimiK2Parser, LlamaParser, MinimaxM2Parser, MistralParser, PassthroughParser,
PythonicParser, QwenParser, QwenXmlParser, Step3Parser,
},
traits::ToolParser,
Expand Down Expand Up @@ -318,8 +318,8 @@ impl ParserFactory {
registry.register_parser("deepseek31", || Box::new(DeepSeek31Parser::new()));
registry.register_parser("deepseek32", || Box::new(DeepSeekDsmlParser::v32()));
registry.register_parser("deepseek_v4", || Box::new(DeepSeekDsmlParser::v4()));
registry.register_parser("glm45_moe", || Box::new(Glm4MoeParser::glm45()));
registry.register_parser("glm47_moe", || Box::new(Glm4MoeParser::glm47()));
registry.register_parser("glm", || Box::new(GlmParser::default()));
registry.register_parser("glm45", || Box::new(GlmParser::glm45()));
Comment on lines +321 to +322

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the old GLM parser aliases

Deployments that explicitly configure the previously registered parser names (--tool-call-parser glm45_moe or glm47_moe) now fail startup because AppContextBuilder validates the configured name with factory.has_parser(name) and rejects unknown tool-call parsers. This change only registers the new glm/glm45 names, so existing configs that worked before this refactor cannot start unless they are updated; keeping the old names as aliases would avoid that compatibility break.

Useful? React with 👍 / 👎.

registry.register_parser("step3", || Box::new(Step3Parser::new()));
registry.register_parser_with_structural_tag(
"kimik2",
Expand Down Expand Up @@ -386,11 +386,10 @@ impl ParserFactory {
registry.map_model("deepseek-ai/DeepSeek-V4*", "deepseek_v4");
registry.map_model("deepseek-*", "pythonic");

// GLM models
registry.map_model("glm-4.5*", "glm45_moe");
registry.map_model("glm-4.6*", "glm45_moe");
registry.map_model("glm-4.7*", "glm47_moe");
registry.map_model("glm-*", "json");
// GLM models (4.5/4.6 use newline format, 4.7+ uses whitespace-only format)
registry.map_model("glm-4.5*", "glm45");
registry.map_model("glm-4.6*", "glm45");
registry.map_model("glm-*", "glm");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep legacy GLM models on JSON parser

This catch-all now routes every glm-* model, including older IDs such as glm-4/glm-4-air that previously fell through to the explicit JSON fallback, into GlmParser. That parser only extracts calls when the response contains <tool_call> markers (GlmParser::parse_complete returns no calls otherwise), so JSON-formatted tool calls from those older GLM models stop being parsed. Since the new parser is documented as covering GLM-4.5 through GLM-5.1, keep a narrower mapping for the native XML families and preserve the JSON fallback for the rest.

Useful? React with 👍 / 👎.


// Step3 models
registry.map_model("step3*", "step3");
Expand Down
2 changes: 1 addition & 1 deletion crates/tool_parser/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ mod tests;
// Re-export types used outside this module
pub use factory::{ParserFactory, PooledParser, ToolConstraint};
pub use parsers::{
CohereParser, DeepSeek31Parser, DeepSeekDsmlParser, DeepSeekParser, Glm4MoeParser, JsonParser,
CohereParser, DeepSeek31Parser, DeepSeekDsmlParser, DeepSeekParser, GlmParser, JsonParser,
KimiK2Parser, LlamaParser, MinimaxM2Parser, MistralParser, PythonicParser, QwenParser,
Step3Parser,
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,14 @@ use crate::{
types::{FunctionCall, StreamingParseResult, ToolCall, ToolCallItem},
};

/// GLM-4 MoE format parser for tool calls
/// GLM tool call format parser.
///
/// Handles both GLM-4 MoE and GLM-4.7 MoE formats:
/// - GLM-4: `<tool_call>{name}\n<arg_key>{key}</arg_key>\n<arg_value>{value}</arg_value>\n</tool_call>`
/// - GLM-4.7: `<tool_call>{name}<arg_key>{key}</arg_key><arg_value>{value}</arg_value></tool_call>`
/// Handles the XML-style `<tool_call>` format used by GLM-4.5 through GLM-5.1:
/// - GLM-4.5/4.6: `<tool_call>{name}\n<arg_key>{key}</arg_key>\n<arg_value>{value}</arg_value>\n</tool_call>`
/// - GLM-4.7/5/5.1: `<tool_call>{name}<arg_key>{key}</arg_key><arg_value>{value}</arg_value></tool_call>`
///
/// Features:
/// - XML-style tags for tool calls
/// - Key-value pairs for arguments
/// - Support for multiple sequential tool calls
pub struct Glm4MoeParser {
/// The default constructor uses the 4.7+ format (no newline between function name and args).
pub struct GlmParser {
/// Regex for extracting complete tool calls
tool_call_extractor: Regex,
/// Regex for extracting function details
Expand All @@ -45,19 +42,13 @@ pub struct Glm4MoeParser {
eot_token: &'static str,
}

impl Glm4MoeParser {
/// Create a new generic GLM MoE parser with a custom func_detail_extractor pattern
///
/// # Arguments
/// - `func_detail_pattern`: Regex pattern for extracting function name and arguments
/// - For GLM-4: `r"(?s)<tool_call>([^\n]*)\n(.*)</tool_call>"`
/// - For GLM-4.7: `r"(?s)<tool_call>\s*([^<\s]+)\s*(.*?)</tool_call>"`
impl GlmParser {
/// Create a new GLM parser with a custom func_detail_extractor pattern.
#[expect(
clippy::expect_used,
reason = "regex patterns are compile-time string literals"
)]
pub(crate) fn new(func_detail_pattern: &str) -> Self {
// Use (?s) flag for DOTALL mode to handle newlines
let tool_call_pattern = r"(?s)<tool_call>.*?</tool_call>";
let tool_call_extractor = Regex::new(tool_call_pattern).expect("Valid regex pattern");

Expand All @@ -79,13 +70,14 @@ impl Glm4MoeParser {
}
}

/// Create a new GLM-4.5/4.6 MoE parser (with newline-based format)
/// Create a GLM-4.5/4.6 parser (newline between function name and args).
pub fn glm45() -> Self {
Self::new(r"(?s)<tool_call>([^\n]*)\n(.*)</tool_call>")
}

/// Create a new GLM-4.7 MoE parser (with whitespace-based format)
pub fn glm47() -> Self {
/// Create a GLM-4.7+ parser (no newline required between function name and args).
/// Compatible with GLM-4.7, GLM-5, GLM-5.1.
pub(crate) fn glm47() -> Self {
Self::new(r"(?s)<tool_call>\s*([^<\s]+)\s*(.*?)</tool_call>")
Comment on lines +80 to 81

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the public GLM 4.7 constructor

For downstream Rust users of the published tool-parser crate, this narrows the previously public Glm4MoeParser::glm47() constructor to pub(crate) on the renamed type, so code that explicitly selected the GLM-4.7 parser can no longer compile while glm45() remains public for compatibility. Keeping glm47() public and/or adding a Glm4MoeParser alias would make this a non-breaking refactor for direct crate consumers.

Useful? React with 👍 / 👎.

}

Expand Down Expand Up @@ -172,16 +164,16 @@ impl Glm4MoeParser {
}
}

impl Default for Glm4MoeParser {
impl Default for GlmParser {
fn default() -> Self {
Self::glm45()
Self::glm47()
}
}

#[async_trait]
impl ToolParser for Glm4MoeParser {
impl ToolParser for GlmParser {
async fn parse_complete(&self, text: &str) -> ParserResult<(String, Vec<ToolCall>)> {
// Check if text contains GLM-4 MoE format
// Check if text contains GLM format
if !self.has_tool_markers(text) {
return Ok((text.to_string(), vec![]));
}
Expand Down Expand Up @@ -276,7 +268,7 @@ impl ToolParser for Glm4MoeParser {
tracing::debug!("Invalid tool name '{}' - skipping", tool_call.function.name);
helpers::reset_current_tool_state(
&mut self.buffer,
&mut false, // glm45_moe/glm47_moe doesn't track name_sent per tool
&mut false,
&mut self.streamed_args_for_tool,
&self.prev_tool_call_arr,
);
Expand Down
4 changes: 2 additions & 2 deletions crates/tool_parser/src/parsers/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ pub mod cohere;
pub mod deepseek;
pub mod deepseek31;
pub mod deepseek_dsml;
pub mod glm4_moe;
pub mod glm;
pub mod json;
pub mod kimik2;
pub mod llama;
Expand All @@ -27,7 +27,7 @@ pub use cohere::CohereParser;
pub use deepseek::DeepSeekParser;
pub use deepseek31::DeepSeek31Parser;
pub use deepseek_dsml::DeepSeekDsmlParser;
pub use glm4_moe::Glm4MoeParser;
pub use glm::GlmParser;
pub use json::JsonParser;
pub use kimik2::KimiK2Parser;
pub use llama::LlamaParser;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
//! GLM-4.7 MoE Parser Integration Tests
//! GLM-4.7+ Tool Call Parser Integration Tests (default format)
mod common;

use common::create_test_tools;
use tool_parser::{Glm4MoeParser, ToolParser};
use tool_parser::{GlmParser, ToolParser};

#[tokio::test]
async fn test_glm47_complete_parsing() {
let parser = Glm4MoeParser::glm47();
let parser = GlmParser::default();

let input = r"Let me search for that.
<tool_call>get_weather<arg_key>city</arg_key><arg_value>Beijing</arg_value><arg_key>date</arg_key><arg_value>2024-12-25</arg_value></tool_call>
Expand All @@ -24,7 +24,7 @@ The weather will be...";

#[tokio::test]
async fn test_glm47_multiple_tools() {
let parser = Glm4MoeParser::glm47();
let parser = GlmParser::default();

let input = r"<tool_call>search<arg_key>query</arg_key><arg_value>rust tutorials</arg_value></tool_call><tool_call>translate<arg_key>text</arg_key><arg_value>Hello World</arg_value><arg_key>target_lang</arg_key><arg_value>zh</arg_value></tool_call>";

Expand All @@ -37,7 +37,7 @@ async fn test_glm47_multiple_tools() {

#[tokio::test]
async fn test_glm47_type_conversion() {
let parser = Glm4MoeParser::glm47();
let parser = GlmParser::default();

let input = r"<tool_call>process<arg_key>count</arg_key><arg_value>42</arg_value><arg_key>rate</arg_key><arg_value>1.5</arg_value><arg_key>enabled</arg_key><arg_value>true</arg_value><arg_key>data</arg_key><arg_value>null</arg_value><arg_key>text</arg_key><arg_value>string value</arg_value></tool_call>";

Expand All @@ -55,7 +55,7 @@ async fn test_glm47_type_conversion() {

#[tokio::test]
async fn test_glm47_streaming() {
let mut parser = Glm4MoeParser::glm47();
let mut parser = GlmParser::default();

let tools = create_test_tools();

Expand Down Expand Up @@ -88,7 +88,7 @@ async fn test_glm47_streaming() {

#[test]
fn test_glm47_format_detection() {
let parser = Glm4MoeParser::glm47();
let parser = GlmParser::default();

// Should detect GLM-4 format
assert!(parser.has_tool_markers("<tool_call>"));
Expand All @@ -102,7 +102,7 @@ fn test_glm47_format_detection() {

#[tokio::test]
async fn test_python_literals() {
let parser = Glm4MoeParser::glm47();
let parser = GlmParser::default();

let input = r"<tool_call>test_func<arg_key>bool_true</arg_key><arg_value>True</arg_value><arg_key>bool_false</arg_key><arg_value>False</arg_value><arg_key>none_val</arg_key><arg_value>None</arg_value></tool_call>";

Expand All @@ -118,7 +118,7 @@ async fn test_python_literals() {

#[tokio::test]
async fn test_glm47_nested_json_in_arg_values() {
let parser = Glm4MoeParser::glm47();
let parser = GlmParser::default();

let input = r#"<tool_call>process<arg_key>data</arg_key><arg_value>{"nested": {"key": "value"}}</arg_value><arg_key>list</arg_key><arg_value>[1, 2, 3]</arg_value></tool_call>"#;

Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
//! GLM-4 MoE Parser Integration Tests
//! GLM Tool Call Parser Integration Tests
mod common;

use common::create_test_tools;
use tool_parser::{Glm4MoeParser, ToolParser};
use tool_parser::{GlmParser, ToolParser};

#[tokio::test]
async fn test_glm4_complete_parsing() {
let parser = Glm4MoeParser::glm45();
async fn test_glm_complete_parsing() {
let parser = GlmParser::glm45();

let input = r"Let me search for that.
<tool_call>get_weather
Expand All @@ -28,8 +28,8 @@ The weather will be...";
}

#[tokio::test]
async fn test_glm4_multiple_tools() {
let parser = Glm4MoeParser::glm45();
async fn test_glm_multiple_tools() {
let parser = GlmParser::glm45();

let input = r"<tool_call>search
<arg_key>query</arg_key>
Expand All @@ -50,8 +50,8 @@ async fn test_glm4_multiple_tools() {
}

#[tokio::test]
async fn test_glm4_type_conversion() {
let parser = Glm4MoeParser::glm45();
async fn test_glm_type_conversion() {
let parser = GlmParser::glm45();

let input = r"<tool_call>process
<arg_key>count</arg_key>
Expand Down Expand Up @@ -79,8 +79,8 @@ async fn test_glm4_type_conversion() {
}

#[tokio::test]
async fn test_glm4_streaming() {
let mut parser = Glm4MoeParser::glm45();
async fn test_glm_streaming() {
let mut parser = GlmParser::glm45();

let tools = create_test_tools();

Expand Down Expand Up @@ -112,10 +112,10 @@ async fn test_glm4_streaming() {
}

#[test]
fn test_glm4_format_detection() {
let parser = Glm4MoeParser::glm45();
fn test_glm_format_detection() {
let parser = GlmParser::glm45();

// Should detect GLM-4 format
// Should detect GLM format
assert!(parser.has_tool_markers("<tool_call>"));
assert!(parser.has_tool_markers("text with <tool_call> marker"));

Expand All @@ -127,7 +127,7 @@ fn test_glm4_format_detection() {

#[tokio::test]
async fn test_python_literals() {
let parser = Glm4MoeParser::glm45();
let parser = GlmParser::glm45();

let input = r"<tool_call>test_func
<arg_key>bool_true</arg_key>
Expand All @@ -149,8 +149,8 @@ async fn test_python_literals() {
}

#[tokio::test]
async fn test_glm4_nested_json_in_arg_values() {
let parser = Glm4MoeParser::glm45();
async fn test_glm_nested_json_in_arg_values() {
let parser = GlmParser::glm45();

let input = r#"<tool_call>process
<arg_key>data</arg_key>
Expand All @@ -166,3 +166,18 @@ async fn test_glm4_nested_json_in_arg_values() {
assert!(args["data"].is_object());
assert!(args["list"].is_array());
}

#[tokio::test]
async fn test_glm_default_parses_glm47_format() {
let parser = GlmParser::default();

let input =
r"<tool_call>get_weather<arg_key>city</arg_key><arg_value>Tokyo</arg_value></tool_call>";

let (_normal_text, tools) = parser.parse_complete(input).await.unwrap();
assert_eq!(tools.len(), 1);
assert_eq!(tools[0].function.name, "get_weather");

let args: serde_json::Value = serde_json::from_str(&tools[0].function.arguments).unwrap();
assert_eq!(args["city"], "Tokyo");
}
4 changes: 2 additions & 2 deletions docs/concepts/architecture/grpc-pipeline.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,8 +273,8 @@ Qwen3-Coder / Qwen3.5+ XML format with parameter tags.
| `pythonic` | `llama-4*`, `deepseek-*` | Python-style function syntax |
| `llama` | `llama-3.2*` | Python tag with JSON |
| `deepseek` | `deepseek-v3*` | XML with function syntax |
| `glm45_moe` | `glm-4.5*`, `glm-4.6*` | GLM 4.5/4.6 MoE format |
| `glm47_moe` | `glm-4.7*` | GLM 4.7 MoE format |
| `glm` | `glm-*` | GLM 4.7+ format (`glm-4.5*`/`glm-4.6*` are matched to `glm45` first) |
| `glm45` | `glm-4.5*`, `glm-4.6*` | GLM 4.5/4.6 format (newline-based) |
| `step3` | `step3*`, `Step-3*` | Step-3 model format |
| `kimik2` | `kimi-k2*`, `Kimi-K2*` | Kimi K2 model format |
| `minimax_m2` | `minimax*`, `MiniMax*` | MiniMax M2 model format |
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/grpc-workers.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,8 @@ Auto-detected from the model name. Override with `--tool-call-parser` if needed.
| `mistral` | Mistral, Mixtral |
| `qwen` | Qwen |
| `qwen_xml` | Qwen3-Coder, Qwen3.5+ |
| `glm45_moe` | GLM-4.5, GLM-4.6 |
| `glm47_moe` | GLM-4.7 |
| `glm` | GLM-4.7, GLM-5, GLM-5.1 |
| `glm45` | GLM-4.5, GLM-4.6 |
| `step3` | Step-3 |
| `kimik2` | Kimi-K2 |
| `minimax_m2` | MiniMax |
Expand Down
4 changes: 2 additions & 2 deletions model_gateway/benches/tool_parser_benchmark.rs
Original file line number Diff line number Diff line change
Expand Up @@ -291,8 +291,8 @@ fn bench_complete_parsing(c: &mut Criterion) {
("pythonic_multi", "pythonic", PYTHONIC_MULTI),
("deepseek", "deepseek", DEEPSEEK_FORMAT),
("kimik2", "kimik2", KIMIK2_FORMAT),
("glm45", "glm45_moe", GLM45_FORMAT),
("glm47", "glm47_moe", GLM47_FORMAT),
("glm45", "glm45", GLM45_FORMAT),
("glm47", "glm", GLM47_FORMAT),
("step3", "step3", STEP3_FORMAT),
("gpt_oss", "gpt_oss", GPT_OSS_FORMAT),
];
Expand Down
Loading