Tag: grounding
All the articles with the tag "grounding".
Computer-Use Agents: Screens as Observations, the Mouse as a Tool
Updated:A computer-use agent operates software the way a person does: it looks at the screen, decides, and clicks or types. The perceive-think-act loop; screenshots versus accessibility trees, measured on real pages (a screenshot is 1,000 to 4,800 tokens; a raw accessibility tree 4,000 to 170,000, pruned 600 to 20,000); coordinate grounding; the action space; what a task costs as screenshots pile up; OSWorld and how far agents have come; and why safety, not reasoning, decides where these agents can be deployed.