Anthropic Fable 5.2 真实自由职业基准完成率达 41.3%,一年前最佳仅 2.5%
Anthropic's Fable 5.2 completes 41 percent of real freelance jobs, up from 2.5 percent a year ago
- 1Fable 5.2 在真实自由职业基准上完成 41.3% 的任务,一年前最佳系统 Manus 仅 2.5%
- 2榜单次名 Opus 5.5 为 21.25%,GPT 6 Astra 为 20.83%,Fable 5.2 完成量约为对手两倍
- 3基准取自 Upwork 平台 240 个真实项目,总价值约 $144,000,由受训人工专家逐项核验交付质量
- 4Scale Labs 称其进步最明显的是音频编辑、建筑室内设计和 3D 建模,工业产品设计与动画仍几乎无法处理
智能体在真实专业任务上的完成率一年内从 2.5% 升至 41.3%,且由人工在专业软件中核验,含金量高于常规跑分。
- 影响:音频编辑、建筑设计、3D 建模等外包任务已接近半数可由智能体完成,外包预算与人力配置可据此重算;失败案例多为业余水平、半成品或文件损坏,交付质检环节不能省;用 AI 审 AI 在该测试中已告失败,质量把关需保留人工。
- 参考:可考虑从音频编辑、室内设计、3D 建模这些榜单提升领先的类目中挑低风险外包项目试点;值得用基准里的单项均价 $632.60 和人工平均近 29 小时作参照,先算出自有外包盘子的可替代规模;验收时保留人工用 Blender 等专业软件打开成品的步骤。
- 依据:结果来自 Scale Labs 在 X 上公布的榜单与 RLI 研究论文,240 个 Upwork 真实项目由受训人工专家核验,属第三方基准数据而非厂商自评。
The progress shows how fast AI agents are catching up on practical work. When the benchmark launched about a year ago, the top system at the time, Manus, managed just 2.5 percent of tasks. Fable 5.2 now completes more than 16 times as many professional jobs.
The jump is also large compared to the most recent measurements. In July, Fable 5, an older model from the same family, was still in front at about 16 percent. That score was already a big step forward: the best agents had more than quadrupled the share of jobs they completed at professional quality within eight months. Four months later, Fable 5.2 hits 41.3 percent. That's roughly two and a half times Fable 5's July score. Its direct predecessor, Fable 5.1, also trails far behind at just under 18 percent, and Fable 5.2 completes more than twice as many jobs.

The next spots on the current leaderboard go to Anthropic's Opus 5.5 at 21.25 percent and OpenAI's GPT 6 Astra at 20.83 percent. Fable 5.2 works almost twice as successfully as the next-best systems on the market. Google's strongest model in the test, Gemini Argon 4, lands in fifth place at 17.08 percent, while older models like GPT-5.2 or Gemini 3 Pro sit below three percent.
Fable 5.2 breaks through in 3D models, audio, and architecture
Scale Labs said on X that Fable 5.2 makes its biggest gains in editing audio and music, designing architecture and interior spaces, and creating 3D models. The agent also handles long, complex projects that previously top-ranked models regularly failed.
Nearly 50 of Fable 5.2's successful completions were jobs no AI agent in the test had solved before. Scale Labs says the AI can still barely handle other tasks, especially industrial product design and animations. When AI agents fail in the test, they typically deliver amateur-quality work, incomplete results, broken files, or contradictory designs, according to the researchers.
Real jobs worth $144,000
Unlike conventional AI tests that often check only isolated skills, the RLI research paper covers complete work assignments. The dataset includes 240 completed projects from 358 verified professionals on the Upwork platform. Together, these jobs are worth about $144,000.
A single project costs an average of $632.60 and takes a human freelancer an average of just under 29 hours. Tasks range from video editing to CAD design, game development, and data analysis. To work on these jobs, AI agents get access to a virtual workspace with common professional software like Blender, GIMP, or Audacity, along with specialized AI tools for image, audio, and video generation.

Trained human experts manually check whether an AI agent completed a job successfully. They open the submitted files in the relevant professional software and compare the AI's output directly with the work the human freelancer delivered. CAIS's attempts to automate this labor-intensive review with another AI have so far failed. Because AI reviewers can barely operate the professional software themselves, they're fooled by submissions that look polished on the surface but are technically flawed, and they rate the models far too generously.
来源:The Decoder:AI News · the-decoder.com