ScrapeForge

ajazhussainsiddiqui/ScrapeForge

★ 0 stars Python Browser/Web Updated today

A self-learning web scraper that uses LLMs to generate CSS extraction schemas once per page template, then caches and auto-generalizes URL patterns. Zero manual selectors. Zero maintenance. FastAPI + MCP server included.

View on GitHub → 🔍 Audit Wallet Slippage →

Quick Install

Copy the config for your editor. Some servers may need additional setup — check the README.

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "scrapeforge": {
      "command": "uvx",
      "args": [
        "scrapeforge"
      ]
    }
  }
}

Or install with pip: pip install scrapeforge

README Excerpt

> **A web scraper that learns page layouts and remembers them.** > One LLM call per page *template*, not per page. --- ScrapeForge is a **self-learning, LLM-powered web scraper** that doesn't just extract data , it *understands* page templates and caches them for reuse. Traditional scrapers break when a site redesigns. ScrapeForge breaks this cycle.

Tools (10)

CRAWL_TIMEOUT_SECONDSDATABASE_URLDELETEENDPOINT_API_KEYGETPOSTRATE_LIMIT_REQUESTSRATE_LIMIT_WINDOWSCHEMA_FAILURE_SCORE_THRESHOLDSIMILARITY_THRESHOLD

Topics

ai-scraperautomationawscrawl4aidata-extractiondockerfastapiintelligent-scraperllmmcpplaywrightpythonschema-learningscraping-apistructured-data

ScrapeForge

Quick Install

README Excerpt

Tools (10)

Topics

Related Browser/Web Servers