ByteCrawl

abrahamperz/ByteCrawl

★ 11 stars Python AI/LLM Updated today

The focused-crawling MCP server for AI agents. Rank a whole site by your topic, read any page as clean Markdown, extract anything

View on GitHub → 🔍 Audit Wallet Slippage →

Quick Install

Copy the config for your editor. Some servers may need additional setup — check the README.

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "bytecrawl": {
      "command": "uvx",
      "args": [
        "bytecrawl"
      ]
    }
  }
}

Or install with pip: pip install bytecrawl

README Excerpt

*Léelo en [español](README.es.md).* **Give your AI agent focused web crawling.** ByteCrawl is an MCP server (and a small Python library) that doesn't just scrape a page — it crawls a whole site and returns the pages *most relevant* to your topic first, using Shark-Search and OPIC in pure Python. - **Webpage**: https://bytecrawl.vercel.app/

Tools (9)

bfscompare_strategiesextractfetch_json_apifetch_markdownfocused_crawllist_linksopicshark

Topics

aiai-agentscrawlerdata-extractionfocused-crawlinghtml-to-markdownllmmcpmcp-serveropicpythonshark-searchweb-scraping

ByteCrawl

Quick Install

README Excerpt

Tools (9)

Topics

Related AI/LLM Servers