Tag: agents
All the articles with the tag "agents".
Evaluating and Observing Agents: Outcomes, Trajectories, Traces and Spans
Updated:An agent's answer can be right for the wrong reasons, wrong for a trivial one, or right only some of the time. Outcome, trajectory and tool-use evaluation; SWE-bench and τ-bench; pass@k against pass^k, with the arithmetic and unbiased estimators; and a small agent instrumented with OpenTelemetry's GenAI span conventions and evaluated from its traces, where most 'failures' turned out to be the answer's format or the harness's own token budget, and the traces showed exactly which.
Many Agents: Subagents, Communication and Orchestration
Updated:Splitting a task across agents buys clean contexts and parallelism, and costs coordination and lost information. Why subagents exist; roles; communication topologies; the A2A protocol; the standard orchestration patterns; a token-cost model, on real document sizes, of one agent against a lead with six workers (a context a quarter to two-fifths the size and far fewer sequential steps whatever the assumptions, a token bill that can go either way, and only 8% of the source reaching the lead); and the published evidence on when multi-agent systems win and when they lose.
Giving Agents Tools and Knowledge: MCP, Skills and OKF
Updated:Three answers to one question, how an agent gets capabilities without bloating its context: the Model Context Protocol for live tools and data, Skills for procedures loaded on demand, and the Open Knowledge Format for curated knowledge. A working MCP server for this blog, called through the SDK and as raw JSON-RPC under the stateless 2026-07-28 protocol; progressive disclosure's token arithmetic; an OKF bundle generated from the posts; and the security model all three share.