# Crawl API Documentation

Base URL: `https://api-crawl.top1.ai`

## Authentication

Include one of these headers in every paid request:

```text
X-API-Key: sk_live_<YOUR_KEY>
# OR
Authorization: Bearer sk_live_<YOUR_KEY>
```

Create and manage API keys at https://crawl.top1.ai/dashboard/keys.

## Errors

- `400`: Invalid request body or URL
- `401`: Invalid or deactivated API key
- `402`: Insufficient credits
- `403`: Banned or insufficient permissions
- `429`: Rate limit exceeded; retry with exponential backoff
- `5xx`: Temporary API error; retry shortly

## Idempotency

Mutating requests can include `Idempotency-Key: <uuid>` to avoid duplicate charges. Keys are scoped per-API-key and retained for 24 hours.

---

### POST /v1/scrape

Scrape a single page and convert it to Markdown.

**Cost:** 1 credit (3 with extract_schema)


**Request body**

```json
{
  "url": "https://example.com",
  "formats": [
    "markdown"
  ],
  "only_main_content": true,
  "max_chars": 5000
}
```

**cURL example**

```bash
curl -X POST https://api-crawl.top1.ai/v1/scrape \
  -H "X-API-Key: sk_live_<YOUR_KEY>" \
  -H "Content-Type: application/json" \
  -d '{\n  "url": "https://example.com",\n  "formats": [\n    "markdown"\n  ],\n  "only_main_content": true,\n  "max_chars": 5000\n}'
```

### POST /v1/extract

Scrape a page, then extract structured fields with an LLM.

**Cost:** 3 credits


**Request body**

```json
{
  "url": "https://example.com",
  "schema": {
    "title": "string",
    "description": "string"
  }
}
```

**cURL example**

```bash
curl -X POST https://api-crawl.top1.ai/v1/extract \
  -H "X-API-Key: sk_live_<YOUR_KEY>" \
  -H "Content-Type: application/json" \
  -d '{\n  "url": "https://example.com",\n  "schema": {\n    "title": "string",\n    "description": "string"\n  }\n}'
```

### POST /v1/crawl

Deep-crawl a single domain; billed per page actually fetched.

**Cost:** 1 credit / page


**Request body**

```json
{
  "url": "https://example.com",
  "max_pages": 10,
  "only_main_content": true,
  "max_chars_total": 20000
}
```

**cURL example**

```bash
curl -X POST https://api-crawl.top1.ai/v1/crawl \
  -H "X-API-Key: sk_live_<YOUR_KEY>" \
  -H "Content-Type: application/json" \
  -d '{\n  "url": "https://example.com",\n  "max_pages": 10,\n  "only_main_content": true,\n  "max_chars_total": 20000\n}'
```

### POST /v1/search

Search the web and scrape the result pages as Markdown.

**Cost:** 2 credits


**Request body**

```json
{
  "query": "Crawl4AI tutorial",
  "limit": 5,
  "scrape_results": true
}
```

**cURL example**

```bash
curl -X POST https://api-crawl.top1.ai/v1/search \
  -H "X-API-Key: sk_live_<YOUR_KEY>" \
  -H "Content-Type: application/json" \
  -d '{\n  "query": "Crawl4AI tutorial",\n  "limit": 5,\n  "scrape_results": true\n}'
```

### GET /v1/usage

Check remaining credits and usage.

**Cost:** 0 credits


**cURL example**

```bash
curl -H "X-API-Key: sk_live_<YOUR_KEY>" https://api-crawl.top1.ai/v1/usage
```

---

## Links

- OpenAPI / Swagger UI: https://api-crawl.top1.ai/docs
- Interactive docs: https://crawl.top1.ai/docs
- Concise LLM overview: https://crawl.top1.ai/llms.txt