Link: New paper: Emerging trends in AI agent evaluation vis AI as Normal Technology

Sayash Kapoor, Arvind Narayanan, et al. synthesize AI agent evaluation trends across best practices (e.g., metrics beyond accuracy, LLM judges and monitors, evaluation infrastructure), research directions (e.g observational usage data, domain-specific evaluation, security) and practical considerations (e.g. who evaluates, labs shaping evaluations, funding) in Emerging Trends in AI Agent Evaluation.

“Agent evaluation is much more complex than model evaluation”:

And when agents perform better or worse, it is harder to attribute these changes to different components of the agent. Addressing these concerns requires significant engineering effort, and the work is getting increasingly expensive—

To investigate the recent OpenAI-HuggingFace incident after the fact, evaluators relied upon other agents to look for and identify instances of harmful action, an approach they acknowledged carried its own limitations and biases.

“Evidence about agents is needed to understand their effects at different layers, but does not always translate across layers”:

An agent might demonstrate capability in the lab, but face limits when placed in a specific use case. Consider METR’s randomized controlled trial which found that, despite expectations, usage of agent tools toward software development actually increased overall completion time on the tasks at hand. Agents may be able to perform certain tasks, but how that capability translates for humans in the real world requires evaluation in context.

“Who gets to evaluate and how?”:

As the costs of frontier evaluation keep increasing, the ability of independent evaluators to participate will be increasingly determined by their access to increased funding. Whether independent evaluators have sufficient access to models also impacts their ability to measure and assess the true frontier.

Via: https://www.normaltech.ai/p/new-paper-emerging-trends-in-ai-agent