uipe

dirkknibbe/uipe
★ 0 stars TypeScript 🤖 AI/LLM Updated today
Perception layer for AI agents — fuses visual + structural + temporal data into a UI Scene Graph. Exposed as an MCP server.
View on GitHub →

Quick Install

Copy the config for your editor. Some servers may need additional setup — check the README.

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "uipe": {
      "command": "npx",
      "args": [
        "-y",
        "dirkknibbe/uipe"
      ]
    }
  }
}

README Excerpt

A perception layer for AI agents — human-like understanding of web interfaces. Fuses structural (DOM + a11y tree + CSS), visual (three-tier vision pipeline), and temporal data into a unified **UI Scene Graph** that an LLM can reason over fluently. Exposed as an MCP server so Claude can navigate and inspect any live URL.

Tools (20)

ANTHROPIC_API_KEYOLLAMA_MODELOLLAMA_URLOMNIPARSER_URLactanalyze_visualbackclickclickSelectorcompare_statesdetect_elementsget_affordancesget_console_logsget_network_errorsget_sceneget_screenshothovernavigatepressKeyscroll