Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/CN/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ Lightllm 整合了众多的开源方案的优点,包括但不限于 FasterTran
多模态部署 <tutorial/multimodal>
奖励模型部署 <tutorial/reward_model>
OpenAI 接口使用 <tutorial/openai>
工具调用(Function Calling) <tutorial/function_calling>
思考解析(Reasoning Parser) <tutorial/reasoning_parser>
APIServer 参数详解 <tutorial/api_server_args_zh>
lightllm api介绍 <tutorial/api_param>

Expand Down
287 changes: 287 additions & 0 deletions docs/CN/source/tutorial/function_calling.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,287 @@
.. _function_calling:

工具调用(Function Calling)
============================

LightLLM 支持多种主流模型的工具调用功能,提供 OpenAI 兼容的 API。

支持的模型
----------

Qwen2.5/Qwen3
~~~~~~~~~~~~~

**解析器**: ``qwen25``

**格式**:

.. code-block:: xml

<tool_call>
{"name": "function_name", "arguments": {"param": "value"}}
</tool_call>

**启动**:

.. code-block:: bash

python -m lightllm.server.api_server \
--model_dir /path/to/qwen2.5 \
--tool_call_parser qwen25 \
--tp 1

Llama 3.2
~~~~~~~~~

**解析器**: ``llama3``

**格式**: ``<|python_tag|>{"name": "func", "arguments": {...}}``

**启动**:

.. code-block:: bash

python -m lightllm.server.api_server \
--model_dir /path/to/llama-3.2 \
--tool_call_parser llama3 \
--tp 1

Mistral
~~~~~~~

**解析器**: ``mistral``

**格式**: ``[TOOL_CALLS] [{"name": "func", "arguments": {...}}, ...]``

DeepSeek-V3
~~~~~~~~~~~

**解析器**: ``deepseekv3``

**格式**:

.. code-block:: xml

<|tool▁calls▁begin|>
<|tool▁call▁begin|>function<|tool▁sep|>func_name
```json
{"param": "value"}
```
<|tool▁call▁end|>
<|tool▁calls▁end|>

DeepSeek-V3.1
~~~~~~~~~~~~~

**解析器**: ``deepseekv31``

**格式**: 简化的 V3 格式,参数直接内联,无代码块包围

Kimi K2
~~~~~~~

**解析器**: ``kimi_k2``

**格式**:

.. code-block:: xml

<|tool_calls_section_begin|>
<|tool_call_begin|>functions.func_name:0
<|tool_call_argument_begin|>{"param": "value"}
<|tool_call_end|>
<|tool_calls_section_end|>

基本使用
--------

定义工具
~~~~~~~~

.. code-block:: python

tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "获取指定城市的天气信息",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "城市名称"
}
},
"required": ["city"]
}
}
}
]

非流式调用
~~~~~~~~~~

.. code-block:: python

import requests
import json

url = "http://localhost:8088/v1/chat/completions"
data = {
"model": "model_name",
"messages": [
{"role": "user", "content": "北京今天天气怎么样?"}
],
"tools": tools,
"tool_choice": "auto" # "auto" | "none" | "required"
}

response = requests.post(url, json=data).json()
message = response["choices"][0]["message"]

if message.get("tool_calls"):
for tc in message["tool_calls"]:
print(f"工具: {tc['function']['name']}")
print(f"参数: {tc['function']['arguments']}")

流式调用
~~~~~~~~

.. code-block:: python

data = {
"model": "model_name",
"messages": [{"role": "user", "content": "查询北京和上海的天气"}],
"tools": tools,
"stream": True
}

response = requests.post(url, json=data, stream=True)
tool_calls = {}

for line in response.iter_lines():
if line and line.startswith(b"data: "):
chunk = json.loads(line[6:])
delta = chunk["choices"][0]["delta"]

if delta.get("tool_calls"):
for tc in delta["tool_calls"]:
idx = tc.get("index", 0)
if idx not in tool_calls:
tool_calls[idx] = {"function": {"name": "", "arguments": ""}}

if tc["function"].get("name"):
tool_calls[idx]["function"]["name"] = tc["function"]["name"]
if tc["function"].get("arguments"):
tool_calls[idx]["function"]["arguments"] += tc["function"]["arguments"]

多轮对话
~~~~~~~~

.. code-block:: python

# 1. 用户提问
messages = [{"role": "user", "content": "北京天气如何?"}]

# 2. 模型调用工具
response1 = requests.post(url, json={
"messages": messages,
"tools": tools
}).json()

tool_call = response1["choices"][0]["message"]["tool_calls"][0]
messages.append(response1["choices"][0]["message"])

# 3. 返回工具结果
weather_result = {"temperature": 15, "condition": "晴朗"}
messages.append({
"role": "tool",
"tool_call_id": tool_call["id"],
"name": tool_call["function"]["name"],
"content": json.dumps(weather_result, ensure_ascii=False)
})

# 4. 生成最终回答
response2 = requests.post(url, json={"messages": messages}).json()
print(response2["choices"][0]["message"]["content"])

高级功能
--------

并行工具调用
~~~~~~~~~~~~

.. code-block:: python

data = {
"messages": messages,
"tools": tools,
"parallel_tool_calls": True # 启用并行调用
}

强制调用特定工具
~~~~~~~~~~~~~~~~

.. code-block:: python

data = {
"tools": tools,
"tool_choice": {
"type": "function",
"function": {"name": "get_weather"}
}
}

与推理模型集成
~~~~~~~~~~~~~~

.. code-block:: python

data = {
"model": "deepseek-r1",
"tools": tools,
"chat_template_kwargs": {"enable_thinking": True},
"separate_reasoning": True # 分离推理内容
}

response = requests.post(url, json=data).json()
message = response["choices"][0]["message"]

print("推理:", message.get("reasoning_content"))
print("工具调用:", message.get("tool_calls"))

常见问题
--------

**工具调用未触发**
检查 ``--tool_call_parser`` 参数和工具描述是否清晰

**参数解析错误**
确认使用了正确的解析器,检查模型输出格式

**流式模式不完整**
正确处理所有 chunks,使用 ``index`` 字段组装多个工具调用

**与推理模型集成失败**
确保使用最新版本,正确配置 ``separate_reasoning`` 和 ``chat_template_kwargs``

技术细节
--------

**核心文件**:
- ``lightllm/server/function_call_parser.py`` - 解析器实现(1267行)
- ``lightllm/server/api_openai.py`` - API 集成
- ``lightllm/server/build_prompt.py`` - 工具注入
- ``test/test_api/test_openai_api.py`` - 测试示例

**相关 PR**:
- PR #1158: 支持推理内容中的函数调用

参考资料
--------

- OpenAI Function Calling: https://platform.openai.com/docs/guides/function-calling
- JSON Schema: https://json-schema.org/
- LightLLM GitHub: https://github.com/ModelTC/lightllm
Loading