Connect any MCP-compatible AI agent (Cursor, Claude Desktop, Windsurf, Cline, and more) to the Crawl API. Your agent can scrape web pages, extract structured data, deep-crawl domains, run web searches, and check your credit balance in real time.
npm install -g @crawl/mcp
npm package will be available once the first publish completes. If it isn't found yet, use the source build below.
Copy an API key from your Dashboard. The key is only shown once.
Get your API keyCursor → Settings → MCP → Add new MCP server
Claude Desktop → Settings → Developer → Edit Config
Install from the repository directly
git clone https://github.com/winbayai/Crawl.git cd Crawl/mcp-server npm install npm run build npm link
Publish to npm
To enable npm install, create an npm organization @crawl, generate an automation access token, and add it as the NPM_TOKEN secret in GitHub repo settings. CI will publish on the next push to main.
Cursor → Settings → MCP → Add new MCP server
{
"mcpServers": {
"crawl": {
"command": "crawl-mcp",
"args": [],
"env": {
"CRAWL_API_KEY": "sk_live_...",
"CRAWL_API_BASE": "https://api-crawl.top1.ai"
}
}
}
}Claude Desktop → Settings → Developer → Edit Config
{
"mcpServers": {
"crawl": {
"command": "crawl-mcp",
"env": {
"CRAWL_API_KEY": "sk_live_...",
"CRAWL_API_BASE": "https://api-crawl.top1.ai"
}
}
}
}CRAWL_API_KEY=sk_live_... CRAWL_API_BASE=https://api-crawl.top1.ai
scrape
Scrape a single URL to clean Markdown.
extract
Extract structured data via LLM with a JSON schema.
crawl
BFS deep-crawl within the same domain, billed by pages fetched.
search
Web search plus optional per-result scraping.
usage
Check remaining credits and recent usage summary.
Sign up and get 100 free credits plus a default API key.