Anthropic Funds New AI Benchmarks to Enhance Safety and Evaluation

Introduction

In the ever-evolving landscape of artificial intelligence (AI), the need for robust, comprehensive benchmarks to evaluate AI models has become increasingly critical. Addressing this demand, Anthropic, a leading AI research organization, has launched a new program aimed at funding the development of advanced benchmarks. This initiative seeks to evaluate the performance and impact of AI models, including generative models like Anthropic's own Claude, with a focus on AI security and societal implications.

The Program’s Objectives

Anthropic's program, unveiled on Monday, is designed to provide financial support to third-party organizations capable of creating benchmarks that effectively measure advanced AI capabilities. The company aims to elevate the field of AI safety by developing high-quality, safety-relevant evaluations. These benchmarks are intended to address the growing demand for reliable AI assessments, a need that has outpaced the current supply of adequate evaluation tools.

Addressing AI’s Benchmarking Problem

Current AI benchmarks often fail to accurately capture how the average person uses AI systems. Many existing benchmarks, especially those created before the advent of modern generative AI, may no longer effectively measure the capabilities they were designed to assess. Anthropic’s initiative seeks to address these shortcomings by creating new, more relevant benchmarks.

The proposed benchmarks will focus on AI security and societal implications. This includes evaluating a model’s ability to carry out potentially harmful tasks such as cyberattacks, enhancing weapons of mass destruction, and manipulating or deceiving people through means like deepfakes or misinformation. By developing these benchmarks, Anthropic aims to create an "early warning system" for identifying and assessing AI risks, particularly those related to national security and defense.

Supporting Research and Development

Anthropic’s program also intends to support research into benchmarks that probe AI’s potential for aiding scientific studies, conversing in multiple languages, mitigating ingrained biases, and self-censoring toxicity. To achieve these goals, Anthropic envisions new platforms that allow subject-matter experts to develop their own evaluations and conduct large-scale trials involving thousands of users.

The company has hired a full-time coordinator to oversee the program and is open to purchasing or expanding projects with significant scaling potential. Anthropic offers a range of funding options tailored to the needs and stages of each project, allowing teams to interact directly with Anthropic’s domain experts from various relevant fields.

Challenges and Skepticism

While Anthropic’s effort to support new AI benchmarks is commendable, there are concerns about the company’s commercial ambitions in the competitive AI industry. Some experts may be wary of aligning their evaluations with Anthropic's definitions of "safe" or "risky" AI, which could force applicants to conform to the company’s perspective on AI safety.

Additionally, Anthropic's references to catastrophic and deceptive AI risks, such as those involving nuclear weapons, have been met with skepticism. Many experts argue that there is little evidence to suggest that AI will develop world-ending, human-outsmarting capabilities in the near future. These claims could divert attention from pressing regulatory issues like AI’s tendency to produce hallucinations.

Conclusion

Anthropic’s new initiative represents a significant step towards developing comprehensive AI benchmarks that prioritize safety and societal implications. By funding third-party organizations and supporting research into advanced AI evaluations, Anthropic aims to set industry standards for AI assessment. However, the success of this program will depend on its ability to garner trust and cooperation from the broader AI community, balancing commercial interests with the goal of fostering safer and more reliable AI systems.

In its blog post, Anthropic expressed hope that its program would serve as “a catalyst for progress towards a future where comprehensive AI evaluation is an industry standard.” Whether or not this vision becomes a reality, Anthropic's initiative highlights the critical need for robust AI benchmarks in an increasingly AI-driven world.