computer-use

Solid

Use Orca's computer-use CLI to inspect and operate local desktop app windows through accessibility trees, screenshots, and safe UI actions. Use for desktop app interaction: list apps/windows, get app state, read visible UI, click controls, type, press keys, scroll, drag, set values, or perform accessibility actions. Also use for browser windows, webviews, Orca app UI, or other desktop UI. Triggers include "computer use", "orca computer", "read Spotify", "read Slack", "control/click/read in a desktop app", and "get app state".

Web & Frontend 4,656 stars 311 forks Updated today MIT

Install

View on GitHub

Quality Score: 92/100

Stars 20%
100
Recency 20%
100
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
50
License 10%
100
Description 5%
100

Skill Content

# Computer Use Use this skill for desktop UI through `orca computer`. When the requested target is a website or web app, operate the desktop browser app/window that contains the page. ## Preconditions - Prefer `orca computer ...`; on Linux, use `orca-ide computer ...` if `orca` is unavailable. In this Orca worktree, use `./config/scripts/orca-dev computer ...` only when testing the local dev runtime. - Prefer `--json`. Screenshot bytes are omitted from JSON and written to `screenshot.path`. - Do not push, submit forms, send messages, buy items, delete data, change account settings, or expose secrets unless the user explicitly asked for that action. - If an app contains sensitive content, read only what the user requested. ```bash orca status --json orca computer capabilities --json ``` ## Core Loop ```bash orca computer list-apps --json orca computer get-app-state --app com.spotify.client --json orca computer click --app com.spotify.client --element-index 42 --json ``` Use the fresh state returned by each action for the next element index. Element indexes are the numeric labels shown in the tree; they may be sparse when noisy sections are omitted, so never infer valid indexes from `elementCount` or "Visible elements." Element indexes are short-lived and go stale after delays, navigation, focus changes, scrolling, window changes, or app re-rendering. ## App Selectors Prefer bundle IDs from `list-apps`; names are acceptable when unambiguous. Use `pid:<number>` only whe...

Details

Author
stablyai
Repository
stablyai/orca
Created
2 months ago
Last Updated
today
Language
TypeScript
License
MIT

Similar Skills

Semantically similar based on skill content — not just same category