OSScreenObserver

BillJr99/OSScreenObserver

★ 0 stars Python 🤖 AI/LLM Updated 3d ago

Exposes the OS accessibility tree, OCR text, and Claude Vision screen descriptions to AI agents and humans simultaneously, via a Flask web inspector and an MCP stdio server compatible with Claude Desktop and Claude Code.

View on GitHub →

Quick Install

Copy the config for your editor. Some servers may need additional setup — check the README.

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "osscreenobserver": {
      "command": "uvx",
      "args": [
        "osscreenobserver"
      ]
    }
  }
}

Or install with pip: pip install osscreenobserver

README Excerpt

A prototype that exposes the operating system's UI accessibility tree, textual descriptions, and ASCII spatial sketches through two simultaneous interfaces: - **Web inspector** (human-facing) — a browser-based dashboard at `localhost:5001` - **MCP server** (AI-facing) — a stdio MCP server compatible with Claude Desktop and Claude Code

Tools (17)

GETPOSTbring_to_foregroundclick_atget_full_screenshotget_screen_descriptionget_screen_sketchget_screenshotget_visible_areasget_window_structurelist_windowsmsspress_keyscrolltype_textwin32guiwindow_index

OSScreenObserver

Quick Install

README Excerpt

Tools (17)

Related AI/LLM Servers