Platform · Computer Use Agents
Agents that click, type and scroll. The screenshots stay on your servers.
Some work still lives behind screens: an older desktop package with no API, an internal portal, a form that has to be filled the same way every day. A computer-use agent looks at a screenshot, decides what a person would do next, and asks your program to do it. With ZenithAI the model that reads the screen runs on your own servers, and your program performs every action. Nothing runs unless your code runs it.
Live on our servers since 25-10-2026, with held keys and action lists since 26-10-2026. It isn’t in the 4.6.0 installer: it ships with the next installer, and a server gets it when it’s updated.
How it works
A loop your program owns. ZenithAI only proposes.
ZenithAI has no computer or browser of its own for this, and it never touches a mouse or keyboard. Your program owns the screen, whether it is a desktop, a virtual machine or a browser. It decides what is allowed, and when to stop.
- Your program takes a screenshot and sends it with the task to your ZenithAI server.
- ZenithAI proposes the next action, such as a click at a point, a key combination, text to type or a scroll. With the built-in
computertool it can propose up to five steps that need no new look at the screen, such as a click on a field and the text to type. - Your program performs it, after any check or confirmation you build in.
- A new screenshot goes back, and the loop continues.
- The loop ends when the model answers in plain text, or when your program reaches its step limit.
Two ways to build it
Bring existing agent code, or define your own actions.
Both run through ZenithAI’s OpenAI-compatible /v1 API, so the usual client libraries work after you change the server address and the model name.
| Built-in computer tool | Your own function tools | |
|---|---|---|
| What you send | The computer_use_preview or computer tool, as existing computer-use code already does | Your own tools, such as click, type_text and scroll |
| Endpoint | POST /v1/responses | POST /v1/chat/completions POST /v1/responses |
| Tier | Genius | Smart or Genius |
| Positions you receive | Screen pixels, converted on the server | A 0 to 1000 grid of the screenshot, which your code maps to the screen |
| Steps per turn | One, or up to five in order | One call per turn |
| Best for | Code and agent frameworks written for the computer tool | Chat Completions code, the Smart tier, or actions of your own |
How it looks to a developer
A screenshot in. A click in pixels, out.
Declare the size of the screen your program clicks on, send the task with a screenshot, and act on the computer_call that comes back. The full loop, with Python and Node code, is in the developer guide.
curl https://your-server/v1/responses \
-H "Authorization: Bearer $API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "Genius",
"truncation": "auto",
"tools": [{"type": "computer_use_preview",
"display_width": 1920,
"display_height": 1080,
"environment": "windows"}],
"input": [{"role": "user", "content": [
{"type": "input_text",
"text": "Click Sign in."},
{"type": "input_image",
"image_url": "data:image/png;..."}]}]
}'
{
"type": "computer_call",
"call_id": "call_1a2b3c...",
"pending_safety_checks": [],
"action": {
"type": "click",
"button": "left",
"x": 1748, "y": 58
}
}
// click there, take a screenshot,
// send it back as computer_call_output
clickPoint and press. Left, right, middle, back or forward buttons, and double-click. Modifier keys can be held, so a ctrl-click or shift-click works.typeText and keys. Text for the focused field, and key combinations such as Ctrl+S or Enter.scrollMove around the page. Scrolling at a point, a drag along a path, or a pointer move to open a hover menu.waitLet the screen catch up. A pause or a fresh screenshot without changing anything, for pages that are still loading.storeHistory, your choice. Send the whole conversation each turn, or ask the server to keep it withstore: trueand continue from the last response. Old screenshots can be dropped automatically withtruncation: "auto".
Where teams put it
Work that only exists on a screen.
When a system has an API, use the API: it is faster and more predictable. A screen agent is for the systems that don’t. These are illustrative patterns, each run in a test environment first and with a person confirming anything that changes data.
| Where | A typical task | Control to keep |
|---|---|---|
| Older desktop software with no API | Open a record and read the figures shown on screen into a report | Read-only: allow clicks and scrolling, block typing |
| Internal web portal | Fill a routine form from a spreadsheet row | A person confirms before Submit |
| Testing your own application | Walk through a sign-up or checkout screen on a test server and report what it saw | Test server and test data only |
| Back-office checks | Compare what two internal screens show for the same customer | A step limit, and an allow-list of applications |
For integrations that call ZenithAI directly rather than through a screen, see API integration use cases and the Enterprise Knowledge API.
Safety is designed in your code
The model proposes. Your program decides.
A screen agent sees whatever is on the screen, and text on a web page or in a document can try to tell it what to do. ZenithAI runs no safety classifier on actions, and says so: the check that matters is the one in your program, before an action runs.
- 01Screenshots stay on your servers · the screen is read by the model on your own ZenithAI deployment. By default nothing is kept after the answer.
- 02Kept only when asked · with
store: truea response and its screenshots are kept on your server, for that API key only, for up to 30 days, and can be deleted at any time. - 03Confirmation is yours · ask a person before payments, messages, deletions, form submissions or anything that enters credentials. When several steps come in one turn, check each of them.
- 04A sandbox, not a daily desktop · run agents in a virtual machine or a spare account with nothing valuable open, with a step limit and a way to abort.
- 05Usage you can see · agent requests count as API use per key, and administrators can switch the API off or tighten its limits.
Before you build
What it does, and what it won’t do.
- Check every step. The shape of every action is guaranteed. Whether it is the right action is the model’s judgement, so a click can still land on the wrong target.
- Keep screenshots small. About 1280 pixels wide is enough, and it keeps more of the history inside the request limits.
- Prefer an API where one exists. A screen agent is slower than a direct call, and screens change between software versions.
| Tiers | Built-in computer tool on Genius. Your own function tools on Smart and Genius. |
|---|---|
| Steps per turn | One. With the built-in computer tool, up to five run in order before the next screenshot. |
| Screenshots | Up to 8 images and 20 MiB per request. truncation: "auto" drops the oldest when a long run no longer fits. |
| History | Sent again each turn, or kept on the server with store: true for 30 days. |
| Requests | 60 a minute per API key by default, across the /v1 API. |
| Accuracy | In our own small test on Genius, 15 of 15 clicks landed within 6 pixels of the target. That is not a benchmark. |
| Safety checks | None on the server. Your program confirms sensitive actions. |
| Availability | Live on our servers since 25-10-2026; not in installer 4.6.0. It ships with the next installer. |
Questions
Computer use agents, answered.
What is computer use in ZenithAI?
/v1 API on your own servers.Does ZenithAI control our computers?
Does existing computer-use code work with ZenithAI?
/v1/responses accepts the computer_use_preview and computer tools and returns computer_call items with positions already in screen pixels. Change the server address and the model name. Agents built on their own click and type functions work on Smart and Genius.Where do the screenshots go?
store: true, it is kept on your server, for that API key only, for up to 30 days, and can be deleted at any time. If you use an agent framework, switch off any tracing that uploads to a third party.Is it safe to let an AI agent operate a screen?
How accurate is it?
Which ZenithAI servers have it?
Developer guide: overview · built-in computer tool · your own function tools · errors, safety and limits
Bring one real task
Pick a screen your team works through every day. Watch an agent try it.
In a demonstration, we show a computer-use agent working through a test screen, with every step shown before it runs, and talk through how it would fit one of your tasks.