MCP computer use lets a coding agent such as Claude Code, Codex or Cursor hand a step it cannot do from the terminal to an agent that can use a screen. Your coding agent keeps the code. The Sai MCP server takes the dashboard setting, desktop app or consent page, does it on a real computer, and returns what it saw. Sensitive steps wait for your approval.
What MCP computer use means for a coding agent
MCP computer use is the pattern where an AI application calls a computer-use agent through the Model Context Protocol. The Model Context Protocol is an open standard for connecting AI applications to external tools and data. A computer-use agent operates a graphical interface by reading the screen, clicking and typing. Together, they let a coding agent delegate steps that exist only in a GUI.
The coding agent stays in charge of the task. It decides that a step needs a screen, describes the step, and reads the result. The computer-use agent only does the GUI part.
Why coding agents stop at GUI-only steps
Coding agents work through files, shells and APIs. Plenty of real tasks include a step outside all three:
- Console settings without a CLI path. A vendor dashboard toggle, a redirect URI, a webhook URL or a feature flag that your tools do not expose.
- Desktop apps. A database GUI, a vendor installer, a license dialog or an app that only has a native window.
- Consent and sign-in screens. OAuth consent pages and admin approvals are designed for a person to click.
- Internal admin tools. Back-office pages built for staff, with no public API.
Without a handoff, the agent writes "now open the console and add this URI" and the task waits for you.
GUI-only steps coding agents hand off most often
These steps come up in everyday development work and are usually done by clicking through a screen. Each one is a good first handoff, because the end state is easy to check.
| Step | Where it happens | What to ask Sai to return |
|---|---|---|
| Add an OAuth redirect URI for a preview URL | Cloud or identity provider console | The list of URIs after saving, the client ID and a screenshot |
| Register a webhook endpoint | A vendor dashboard such as payments or messaging | The endpoint and events it shows, without copying any secret into the chat |
| Turn on a feature, scope or plan setting | A SaaS admin console | The setting before and after, with a screenshot |
| Invite a test user or service account | A team or workspace admin page | The member list with the new role |
| Check a change in a desktop app | A Windows or macOS app window | Screenshots of the screens you name, and any error dialog text |
| Read a value that only shows in the UI | A dashboard, billing page or status page | The value as text, so your agent can use it in code or a test |
Example handoff instructions
Your coding agent writes these for you once it knows Sai is available. Each names the place, the change, what must not change and what to send back.
- "Ask Sai to open the payments dashboard in test mode, add the webhook endpoint https://preview-482.example.dev/api/webhooks for invoice events, ask me before saving, and return the endpoint list."
- "Ask Sai to open our admin console, enable SSO enforcement for the staging workspace only, ask me before saving, and send a before-and-after screenshot."
- "Ask Sai to install the build from dist/ on the Windows computer, open it, sign in with the test account, and report the version shown in Help > About."
- "Ask Sai to open the analytics dashboard, read yesterday's signup count for the staging project, and return the number as text."
Before you hand off a GUI step: a checklist
Use this with any tool, including doing it by hand:
- Check for an API or CLI first. If one exists and your agent can use it, that is faster and easier to review.
- Name the exact place. The app or URL, the project or account, and the object to change.
- State the end state. What should be true after the step, and which fields must not change.
- Mark the sensitive actions. Saving, submitting, deleting and sign-ins should wait for a person.
- Ask for evidence. A screenshot and the values on screen, so the coding agent can verify them against the code.
- Keep secrets out of the chat. Sign in through an approval flow, not by pasting a password into a prompt.
Ways to handle GUI-only steps from a coding agent, compared
| Approach | Setup | Desktop apps | Approvals | Best for |
|---|---|---|---|---|
| Do it by hand | None | Yes | You are the approval | Rare, one-off steps |
| Playwright MCP | One command in Claude Code or Codex, or a settings entry in Cursor | No, it automates a browser | Your coding agent's tool-call prompts | Web pages you can reach from a local browser |
| Anthropic computer use tool | You build the VM or container, action code and agent loop | Yes, in your environment | You write the confirmation step | Teams building their own computer-use product |
| OpenAI computer use | You provide the environment and execute the actions | Browser or desktop, in your harness | You build the controls | Teams building their own harness on the Responses API |
| Sai MCP server | Add the server to your coding agent | Yes, on a cloud computer or your own Mac or Windows machine | Built in for sign-ins, saving, sending and deleting | Handing off GUI steps without building the computer side |
If the step is on a web page and you are watching, Playwright MCP is free, fast and enough. If you need full control of the environment, build on the Anthropic or OpenAI computer use tools.
How to set up MCP computer use with Sai
- Add the Sai MCP server to Claude Code, Codex or Cursor (see the quick start). In Claude Code, servers are added with
claude mcp addand can be shared with a team through a project.mcp.json. Codex reads MCP servers from~/.codex/config.toml. Cursor reads.cursor/mcp.jsonin the project or~/.cursor/mcp.jsonglobally. - Tell your agent when to use it. Add a line to your instructions: use Sai for any step with no CLI or API. The full example is in the Start with a prompt tab above.
- Let the agent describe the step. It sends Sai the place, the change, what must not change and what to return.
- Approve the sensitive part. Sai asks before sign-ins, saving, submitting, sending or deleting. You can approve once, for the whole task, or deny.
- Let the agent verify. Sai returns what it saw, and your coding agent checks it against the code and runs its tests.
Sai ranks #1 on OSWorld, the public benchmark for agents that complete tasks on a real computer. Once Sai completes a task, its steps are kept as code that replays the same way, so the next handoff of the same step is faster and more consistent.
From your coding agent or your own code: the MCP server is the simplest path inside Claude Code, Codex or Cursor. From a script or CI job, call POST /v1/agents/message instead. Paid plans add dedicated computers, one task each, so several handoffs can run at once. Read the quick start.
Handing off safely
- Trust the servers you add. Claude Code's documentation tells you to verify you trust each server, because servers that fetch external content can expose you to prompt injection.
- Treat the screen as untrusted. Anthropic and OpenAI both warn that text on a page can try to override instructions. Keep the handoff narrow and say what must not change.
- Keep approvals on. Cursor asks before using MCP tools by default, and Claude Code asks before using project-scoped servers in interactive sessions. Keep Sai's own approvals for saving and deleting too.
- Use staging accounts. Point handoffs at staging projects and test tenants until the step is routine.
- Check the result in code. Ask your agent to compare what Sai saw with your config, not to take the report on trust.
Related use cases: give your own agent a computer tool and test your signup flow end to end.
Last reviewed October 7, 2026.