AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE
Benchmark Radar.Benchmarks. Model results. Original sources.
/

Tools and agents benchmarks

Distinguish correct tool calls from successful multi-step work, and read results as evaluations of a configured system.

An agent evaluation tests a system that chooses actions, observes their results and continues toward a goal. Tool use can be a small part of that process: selecting a function and supplying valid arguments. A complete task can additionally require planning, remembering earlier observations and recovering when a tool fails. The Berkeley Function Calling Leaderboard separates tool-calling settings, while environments such as WebArena evaluate work performed through an application.

What these benchmarks test

  • Tool selection and argument construction test whether a requested action is expressed correctly.
  • Multi-turn tasks test whether later decisions use earlier outputs and changing state.
  • End-to-end success measures whether the requested outcome was achieved within the allowed environment.

How to read the results

  • Treat the model, system prompt, agent loop, tools and memory policy as part of the evaluated configuration.
  • Separate valid call syntax, correct execution and final task completion; these are not interchangeable metrics.
  • Inspect interaction limits and whether evaluators allow feedback, repair, retries or human assistance.

What to watch for

  • A result attributed to a model may also reflect substantial work done by its surrounding software.
  • More successful tasks can require more time or tool calls; read resource costs alongside completion.

References

  1. Berkeley Function Calling Leaderboard
  2. WebArena Verified

Benchmarks in this topic 301 source records

Search and filter this topic

Related topics

Coding benchmarksComputer use benchmarksCybersecurity benchmarks