跳到正文
原文
通义千问 Qwen:Blog·· 2024-05-11AI 评分49

通义千问发布 Qwen-Max-0428,进入 Chatbot Arena 排行榜前十

Notes on Qwen-Max-0428

文章要点
  • 1通义千问发布 Qwen-Max-0428,一款面向对话服务的指令微调模型
  • 2模型上线 Chatbot Arena 后进入排行榜前十,文中 Arena 分数为 1186
  • 3官方自测 MT-Bench 得分 8.96,超过 Qwen1.5-110B-Chat 的 8.88
  • 4DashScope API 已兼容 OpenAI 格式,网页服务与 App 仅限中国大陆访问
决策要点

通义以新模型 Qwen-Max-0428 进入 Chatbot Arena 前十,把对话模型效果交给第三方榜单检验,同时开放 OpenAI 兼容接口。

  • 影响:企业选型国内对话模型多了一个进入国际榜单前十的候选;DashScope 兼容 OpenAI 格式,已有 OpenAI SDK 的应用迁移成本较低;现有分数为官方口径,接入前宜按自身场景实测。
  • 参考:可考虑用 DashScope 的 OpenAI 兼容接口做小范围接入测试,用自有业务问题集对比 Qwen1.5-110B-Chat 等现有模型;值得先在 Hugging Face Spaces 演示中跑通典型用例再定预算。
  • 依据:MT-Bench 8.96 为官方自测口径,暂无独立验证;前十与 Arena 分数 1186 为 Qwen 团队引述 Chatbot Arena 榜单结果,出自官方博客。
正文

API DEMO DISCORD

Previously, we opensourced a series of Qwen1.5 model ranging from 0.5 to 110 billion parameters. Now, we release a larger model, Qwen-Max-0428. Qwen-Max-0428 is an instruction-tuned model for chat service. Very recently, it is available via Chatbot Arena and it has now become the top-10 in the leaderboard. Furthermore, our evaluation of MT-Bench also demonstrates that the new model outperforms our previous largest model Qwen1.5-110B-Chat.

ModelsMT-BenchArena
Qwen1.5-110B-Chat8.881172
Qwen-Max-04288.961186

We provide a demo of chat service of Qwen-Max-0428 (link) in Hugging Face Spaces:

It is also accessible through the official DashScope API (link). Additionally, now the DashScope API is compatible with the OpenAI API format. Below is an example of usage:

from openai import OpenAI

client = OpenAI(
    api_key="$your-dashscope-api-key",
    base_url="https://dashscope.aliyuncs.com/compatible-mode/v1"
)

completion = client.chat.completions.create(
    model="qwen-max",
    messages=[{'role': 'system', 'content': 'You are a helpful assistant.'},
              {'role': 'user', 'content': 'Tell me something about large language models.'}]
)
print(completion.choices[0].message)

This model is available in our web service and APP (link, only accessible in mainland China). Enjoy!

Citation

@misc{qwen1.5,
    title = {Introducing Qwen1.5},
    url = {https://qwenlm.github.io/blog/qwen1.5/},
    author = {Qwen Team},
    month = {February},
    year = {2024}
}

来源:通义千问 Qwen:Blog · qwenlm.github.io