工具调用（Function Calling）

工具调用（Function Calling）#

LightLLM 支持多种主流模型的工具调用功能，提供 OpenAI 兼容的 API。

支持的模型#

Qwen2.5/Qwen3#

解析器: qwen25

格式:

<tool_call>
{"name": "function_name", "arguments": {"param": "value"}}
</tool_call>

启动:

python -m lightllm.server.api_server \
    --model_dir /path/to/qwen2.5 \
    --tool_call_parser qwen25 \
    --tp 1

Llama 3.2#

解析器: llama3

格式: <|python_tag|>{"name": "func", "arguments": {...}}

启动:

python -m lightllm.server.api_server \
    --model_dir /path/to/llama-3.2 \
    --tool_call_parser llama3 \
    --tp 1

Mistral#

解析器: mistral

格式: [TOOL_CALLS] [{"name": "func", "arguments": {...}}, ...]

DeepSeek-V3#

解析器: deepseekv3

格式:

<｜tool▁calls▁begin｜>
<｜tool▁call▁begin｜>function<｜tool▁sep｜>func_name
```json
{"param": "value"}
```
<｜tool▁call▁end｜>
<｜tool▁calls▁end｜>

DeepSeek-V3.1#

解析器: deepseekv31

格式: 简化的 V3 格式，参数直接内联，无代码块包围

基本使用#

定义工具#

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "获取指定城市的天气信息",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {
                        "type": "string",
                        "description": "城市名称"
                    }
                },
                "required": ["city"]
            }
        }
    }
]

非流式调用#

import requests
import json

url = "http://localhost:8088/v1/chat/completions"
data = {
    "model": "model_name",
    "messages": [
        {"role": "user", "content": "北京今天天气怎么样？"}
    ],
    "tools": tools,
    "tool_choice": "auto"  # "auto" | "none" | "required"
}

response = requests.post(url, json=data).json()
message = response["choices"][0]["message"]

if message.get("tool_calls"):
    for tc in message["tool_calls"]:
        print(f"工具: {tc['function']['name']}")
        print(f"参数: {tc['function']['arguments']}")

流式调用#

data = {
    "model": "model_name",
    "messages": [{"role": "user", "content": "查询北京和上海的天气"}],
    "tools": tools,
    "stream": True
}

response = requests.post(url, json=data, stream=True)
tool_calls = {}

for line in response.iter_lines():
    if line and line.startswith(b"data: "):
        chunk = json.loads(line[6:])
        delta = chunk["choices"][0]["delta"]

        if delta.get("tool_calls"):
            for tc in delta["tool_calls"]:
                idx = tc.get("index", 0)
                if idx not in tool_calls:
                    tool_calls[idx] = {"function": {"name": "", "arguments": ""}}

                if tc["function"].get("name"):
                    tool_calls[idx]["function"]["name"] = tc["function"]["name"]
                if tc["function"].get("arguments"):
                    tool_calls[idx]["function"]["arguments"] += tc["function"]["arguments"]

多轮对话#

# 1. 用户提问
messages = [{"role": "user", "content": "北京天气如何？"}]

# 2. 模型调用工具
response1 = requests.post(url, json={
    "messages": messages,
    "tools": tools
}).json()

tool_call = response1["choices"][0]["message"]["tool_calls"][0]
messages.append(response1["choices"][0]["message"])

# 3. 返回工具结果
weather_result = {"temperature": 15, "condition": "晴朗"}
messages.append({
    "role": "tool",
    "tool_call_id": tool_call["id"],
    "name": tool_call["function"]["name"],
    "content": json.dumps(weather_result, ensure_ascii=False)
})

# 4. 生成最终回答
response2 = requests.post(url, json={"messages": messages}).json()
print(response2["choices"][0]["message"]["content"])

高级功能#

并行工具调用#

data = {
    "messages": messages,
    "tools": tools,
    "parallel_tool_calls": True  # 启用并行调用
}

强制调用特定工具#

data = {
    "tools": tools,
    "tool_choice": {
        "type": "function",
        "function": {"name": "get_weather"}
    }
}

与推理模型集成#

data = {
    "model": "deepseek-r1",
    "tools": tools,
    "chat_template_kwargs": {"enable_thinking": True},
    "separate_reasoning": True  # 分离推理内容
}

response = requests.post(url, json=data).json()
message = response["choices"][0]["message"]

print("推理:", message.get("reasoning_content"))
print("工具调用:", message.get("tool_calls"))

常见问题#

工具调用未触发

检查 --tool_call_parser 参数和工具描述是否清晰

参数解析错误

确认使用了正确的解析器，检查模型输出格式

流式模式不完整

正确处理所有 chunks，使用 index 字段组装多个工具调用

与推理模型集成失败

确保使用最新版本，正确配置 separate_reasoning 和 chat_template_kwargs

技术细节#

核心文件: - lightllm/server/function_call_parser.py - 解析器实现 - lightllm/server/api_openai.py - API 集成 - lightllm/server/build_prompt.py - 工具注入 - test/test_api/test_openai_api.py - 测试示例

相关 PR: - PR #1158: 支持推理内容中的函数调用

参考资料#