A customizable MCP server framework for crawling websites, extracting structured content, and providing powerful search and retrieval capabilities to AI agents.
Quick Install
Copy the config for your editor. Some servers may need additional setup — check the README.
Claude Desktop
Claude Code
Cursor
Add to claude_desktop_config.json:
{
"mcpServers": {
"mcp-server-crawler": {
"command": "npx",
"args": [
"-y",
"codecracker2020/mcp-server-crawler"
]
}
}
}📋 Copy
Run in terminal:
claude mcp add mcp-server-crawler npx -y codecracker2020/mcp-server-crawler📋 Copy
Add to .cursor/mcp.json:
{
"mcpServers": {
"mcp-server-crawler": {
"command": "npx",
"args": [
"-y",
"codecracker2020/mcp-server-crawler"
]
}
}
}📋 Copy