Hands: A Rust MCP Agent That Controls Real Windows Desktops and Chrome

A developer has released Hands, a Rust-based MCP/CLI agent that gives LLM coding harnesses direct control over a Windows desktop and a real Chrome browser profile. Unlike Playwright or Puppeteer-style automation, Hands drives the OS directly, which means it works inside your actual logged-in Chrome session, not a detached automation browser.

Why it matters

Most browser automation tools for AI agents require launching a controlled browser instance, which breaks session state, cookies, extensions, and anything that depends on a real user profile. Hands sidesteps this entirely by operating at the OS input layer.

"I wanted a coding agent to use this Windows PC and a real Chrome profile the way I do: look at the screen, move the real mouse, type, click, without turning Chrome into an automation browser."

The observe tool returns a screenshot path plus a structured element list built from Windows UIA (UI Automation) and optional Chrome DOM IDs. The click tool fires OS SendInput along a Bézier curve path, which looks like human movement to anti-bot systems. This is a meaningful architectural choice, not a cosmetic one.

What changes in practice

• MCP-compatible harnesses (Claude Code, Codex, Grok, OpenCode) can call Hands tools with no custom glue code
• Agents can interact with any visible UI element on Windows, not just web content
• Session continuity is preserved: the agent uses your actual browser state, logins, and extensions
• Anti-automation detection is harder to trigger because input is OS-native, not injected via DevTools
• The tool surface is minimal: observe, click, type, scroll covers most real-world tasks

How to use it

1. Install the Rust binary and configure it as an MCP server in your harness of choice
2. Use observe at the start of each agent step to get a fresh screenshot and element list, ground the LLM in current screen state before acting
3. Prefer UIA element IDs over coordinate-based clicks when available; they are more robust across window resizes
4. For browser automation tasks that require your real session (OAuth flows, internal tools, SaaS dashboards), route through Hands rather than spinning up a headless browser
5. Log every observe output during development; the element list is your primary debugging surface when an agent misclicks

Hands is early-stage and Windows-only for now, so Linux and macOS teams will need to wait or contribute. But the core idea, treating the desktop as a first-class agent interface via MCP, is the right abstraction.

If you are building agents that need to operate on real user machines rather than clean automation environments, Hands is worth evaluating today.