通义千问发布 Qwen-Max-0428,进入 Chatbot Arena 排行榜前十
Notes on Qwen-Max-0428
- 1通义千问发布 Qwen-Max-0428,一款面向对话服务的指令微调模型
- 2模型上线 Chatbot Arena 后进入排行榜前十,文中 Arena 分数为 1186
- 3官方自测 MT-Bench 得分 8.96,超过 Qwen1.5-110B-Chat 的 8.88
- 4DashScope API 已兼容 OpenAI 格式,网页服务与 App 仅限中国大陆访问
通义以新模型 Qwen-Max-0428 进入 Chatbot Arena 前十,把对话模型效果交给第三方榜单检验,同时开放 OpenAI 兼容接口。
- 影响:企业选型国内对话模型多了一个进入国际榜单前十的候选;DashScope 兼容 OpenAI 格式,已有 OpenAI SDK 的应用迁移成本较低;现有分数为官方口径,接入前宜按自身场景实测。
- 参考:可考虑用 DashScope 的 OpenAI 兼容接口做小范围接入测试,用自有业务问题集对比 Qwen1.5-110B-Chat 等现有模型;值得先在 Hugging Face Spaces 演示中跑通典型用例再定预算。
- 依据:MT-Bench 8.96 为官方自测口径,暂无独立验证;前十与 Arena 分数 1186 为 Qwen 团队引述 Chatbot Arena 榜单结果,出自官方博客。
Previously, we opensourced a series of Qwen1.5 model ranging from 0.5 to 110 billion parameters. Now, we release a larger model, Qwen-Max-0428. Qwen-Max-0428 is an instruction-tuned model for chat service. Very recently, it is available via Chatbot Arena and it has now become the top-10 in the leaderboard. Furthermore, our evaluation of MT-Bench also demonstrates that the new model outperforms our previous largest model Qwen1.5-110B-Chat.

| Models | MT-Bench | Arena |
|---|---|---|
| Qwen1.5-110B-Chat | 8.88 | 1172 |
| Qwen-Max-0428 | 8.96 | 1186 |
We provide a demo of chat service of Qwen-Max-0428 (link) in Hugging Face Spaces:
It is also accessible through the official DashScope API (link). Additionally, now the DashScope API is compatible with the OpenAI API format. Below is an example of usage:
from openai import OpenAI
client = OpenAI(
api_key="$your-dashscope-api-key",
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1"
)
completion = client.chat.completions.create(
model="qwen-max",
messages=[{'role': 'system', 'content': 'You are a helpful assistant.'},
{'role': 'user', 'content': 'Tell me something about large language models.'}]
)
print(completion.choices[0].message)
This model is available in our web service and APP (link, only accessible in mainland China). Enjoy!
Citation
@misc{qwen1.5,
title = {Introducing Qwen1.5},
url = {https://qwenlm.github.io/blog/qwen1.5/},
author = {Qwen Team},
month = {February},
year = {2024}
}
来源:通义千问 Qwen:Blog · qwenlm.github.io