Google DeepMind 试点业界首个双盲 AI 模型评估
Piloting the world's first double-blind AI evaluations
- 1Google DeepMind 试点对专有前沿模型的双盲评估，测试在密码学封闭环境中进行
- 2首个受测对象为一个 Gemini Flash Lite 模型，合作方包括新加坡 AI 安全研究院、OpenMined 等
- 3双盲评估针对基准污染问题，模型提前见过测试题会虚高分数，削弱对能力与安全评估的信任
- 4官方称此前已有零日志协议和合同保障，加入密码学技术保障被视为安全评估的进一步动作
基准污染会虚高模型分数，外部评测的信任保障正从协议和零日志约定走向密码学技术手段。
- 影响：企业选型依赖的公开基准分数可能因污染而虚高，采购参考需多一层判断；该试点目前仅覆盖一个 Gemini Flash Lite 模型，尚未形成普遍的评测方式。
- 参考：可考虑在模型采购中把评估是否防污染、有无第三方机构参与列入问询清单；值得跟踪新加坡 AI 安全研究院等合作方的后续评测结果作为选型参考。
- 依据：口径为 Google DeepMind 官方博客，合作机构与受测模型名单属官方说法，防污染效果暂无独立验证。
August 27, 2026 Responsibility & Safety
William Isaac, Sol Messing and Kristian Lum
Building trust in proprietary model benchmarks using cryptographically secure environments
Imagine a student is set to take a high-stakes exam. If they accidentally peek at the test questions in advance, achieving a perfect score is influenced by this knowledge, making it a meaningless accomplishment. To truly measure what they know, they must have no visibility of the test questions until it's time to take the exam. That is the exact challenge the industry faces when evaluating advanced AI models. If a model has already seen the test questions - a problem known as benchmark contamination - the results can only be trusted to an extent.
Today, we’re introducing the world’s first double-blind evaluation of a proprietary, frontier class AI model, which keeps external evaluations confined to a cryptographic “box” where they can’t be used by models later to optimize performance ahead of testing. We're partnering with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons, to test a Gemini Flash Lite model against confidential benchmarks in a privacy-preserving environment, increasing evaluation integrity.
At Google, we assess our AI systems using a broad spectrum of evaluations throughout model development and deployment, but we don’t rely on internal testing alone. To identify potential blindspots, we work with a diverse group of external partners, including specialized research labs, civil society and national AI Safety and Security Institutes (AISIs), using their unique expertise to stress-test our models.
As AI models become more capable, ensuring the model has not seen the test questions or prompts in advance is critical, as this can skew the results. Policymakers, researchers, and enterprises need to trust that AI benchmarks accurately reflect a model's true capabilities and safety, but if models are able to “peek” at the evaluation questions in advance, it can artificially inflate scores and undermine this trust.
Although zero-logging protocols and rigorous contractual safeguards have long kept external test prompts confidential, incorporating technical and cryptographic safeguards marks a major step forward in secure model evaluation.
How double-blind evaluations work
来源：Google DeepMind：Blog · deepmind.google