AI labs have built their reputations on model benchmarking, a system meant to provide neutral and accurate comparisons between AI products. Yet, Irene Solaiman, chief policy officer at Hugging Face, told a TechCrunch Disrupt panel last week that the methods behind these evaluations are still complex and evolving. She argued that testing must become a more consistent science if the industry hopes to rely on it going forward.
“Trust is such a weighted term,” Solaiman said. “In terms of whether we can trust results, one big part is can you reproduce the results of an evaluation? Do you have the level of access that you need to the model itself—this is where open weights come in in a big way.” Reproducibility has long been a sticky issue in benchmarking and continues to spark debate.
Beyond the validity of any single test, Solaiman noted that comparing one evaluation to another remains difficult. “They are an evolving science, where we don’t know how to meaningfully compare them to each other,” she explained. “They start to get ineffective relatively quickly, and we don’t have as much of a thriving ecosystem as we need.” She added that there is a strong demand for an embedded evaluator ecosystem, but the standards that define an effective evaluator across tasks still require much more uniformity.
The concept of embedded evaluators—neutral third parties placed inside AI organizations—has gained traction in recent months as labs confront a wave of safety incidents that raise concerns about potential harms. Hugging Face itself was once the target of such an incident. Rather than waiting for government regulation to address model safety, many labs have turned to third‑party solutions, though doubts persist about their neutrality and effectiveness. Anthropic’s choice to hire Accenture as its first evaluator drew criticism, with some questioning whether a consulting firm is suited for the role.
Evaluations are shifting from simple model tests to assessments of the entire system surrounding the model, including the harness and the agent. Solaiman explained that the harness—the tool users employ to access the model—significantly influences performance. A weaker model paired with a superior harness can outperform a cutting‑edge model. “Agentic evals are really hard and really expensive,” she said. “We have just more evaluations for models… we don’t have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent because that’s what a lot of folks are getting value out of.”
In short, the current evaluation landscape resembles a free‑for‑all. For benchmarking to remain a useful tool for industry leaders, Solaiman suggests a more uniform, industry‑wide approach may be necessary, ensuring everyone operates from the same playbook.