API Reference

Complete reference for the Scrapper REST API. Base URL: https://api.scrapper.dev/v1

Authentication

All API requests require an API key passed via the X-API-Key header.

HEADER

X-API-Key: your_api_key

Include your API key in every request.

curl

curl https://api.scrapper.dev/v1/tasks \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/tasks', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const data = await res.json();

Scrape

Submit scraping jobs to extract data from websites.

POST

/v1/scrape

Create a new scrape task. Returns a task ID to poll for results.

curl

curl -X POST https://api.scrapper.dev/v1/scrape \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "render_js": false,
    "selectors": [
      {"name": "title", "type": "css", "selector": "h1"},
      {"name": "links", "type": "css", "selector": "a", "attribute": "href", "multiple": true}
    ]
  }'

javascript

const res = await fetch('/api/v1/scrape', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    url: 'https://example.com',
    render_js: false,
    selectors: [
      { name: 'title', type: 'css', selector: 'h1' },
      { name: 'links', type: 'css', selector: 'a', attribute: 'href', multiple: true }
    ]
  })
});
const data = await res.json();
console.log(data.task_id);

Response

json

{
  "task_id": "01HQXYZ...",
  "status": "queued",
  "created_at": "2026-01-15T10:30:00Z"
}

Tasks

Retrieve task status and list all tasks.

GET

/v1/tasks/:taskId

Get the current status and metadata for a specific task.

curl

curl https://api.scrapper.dev/v1/tasks/01HQXYZ \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/tasks/01HQXYZ', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const task = await res.json();

Response

json

{
  "task_id": "01HQXYZ...",
  "url": "https://example.com",
  "status": "completed",
  "priority": 5,
  "attempts": 1,
  "max_attempts": 3,
  "created_at": "2026-01-15T10:30:00Z",
  "completed_at": "2026-01-15T10:30:02Z"
}

GET

/v1/tasks

List all tasks with optional status filter and pagination.

curl

curl "https://api.scrapper.dev/v1/tasks?status=completed&page=1&per_page=20" \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/tasks?status=completed&page=1&per_page=20', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const { tasks, total, page, per_page } = await res.json();

Response

json

{
  "tasks": [ ... ],
  "total": 150,
  "page": 1,
  "per_page": 20
}

Results

Retrieve the scraping results for a completed task.

GET

/v1/results/:taskId

Get the scraped content, extracted data, timing, and metadata for a task.

curl

curl https://api.scrapper.dev/v1/results/01HQXYZ \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/results/01HQXYZ', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const result = await res.json();
console.log(result.extracted_data);

Response

json

{
  "task_id": "01HQXYZ...",
  "status": "completed",
  "status_code": 200,
  "content_type": "text/html",
  "timing": {
    "total_ms": 1250,
    "network_ms": 800,
    "render_ms": 450
  },
  "extracted_data": {
    "title": "Example Page",
    "links": ["https://example.com/about", "https://example.com/contact"]
  }
}

Templates

Create and manage reusable scraping configurations.

POST

/v1/templates

Create a new scraping template with selectors and settings.

curl

curl -X POST https://api.scrapper.dev/v1/templates \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Blog Posts",
    "url_pattern": "https://blog.example.com/*",
    "method": "GET",
    "render_js": false,
    "selectors": [
      {"name": "title", "type": "css", "selector": "article h1"},
      {"name": "body", "type": "css", "selector": "article .content"}
    ]
  }'

javascript

const res = await fetch('/api/v1/templates', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: 'Blog Posts',
    url_pattern: 'https://blog.example.com/*',
    method: 'GET',
    render_js: false,
    selectors: [
      { name: 'title', type: 'css', selector: 'article h1' },
      { name: 'body', type: 'css', selector: 'article .content' }
    ]
  })
});

GET

/v1/templates

List all templates.

curl

curl https://api.scrapper.dev/v1/templates \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/templates', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const { templates } = await res.json();

DELETE

/v1/templates/:id

Delete a template by ID.

curl

curl -X DELETE https://api.scrapper.dev/v1/templates/tpl_123 \
  -H "X-API-Key: your_api_key"

javascript

await fetch('/api/v1/templates/tpl_123', {
  method: 'DELETE',
  headers: { 'X-API-Key': 'your_api_key' }
});

Schedules

Set up recurring scrape jobs using cron expressions.

POST

/v1/schedules

Create a new scheduled scrape.

curl

curl -X POST https://api.scrapper.dev/v1/schedules \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Daily check",
    "url": "https://example.com",
    "cron_expression": "0 9 * * *",
    "timezone": "UTC",
    "change_detection": true
  }'

javascript

const res = await fetch('/api/v1/schedules', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: 'Daily check',
    url: 'https://example.com',
    cron_expression: '0 9 * * *',
    timezone: 'UTC',
    change_detection: true
  })
});

GET

/v1/schedules

List all schedules.

curl

curl https://api.scrapper.dev/v1/schedules \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/schedules', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const { schedules } = await res.json();

POST

/v1/schedules/:id/pause

Pause a schedule.

curl

curl -X POST https://api.scrapper.dev/v1/schedules/sch_123/pause \
  -H "X-API-Key: your_api_key"

javascript

await fetch('/api/v1/schedules/sch_123/pause', {
  method: 'POST',
  headers: { 'X-API-Key': 'your_api_key' }
});

POST

/v1/schedules/:id/resume

Resume a paused schedule.

curl

curl -X POST https://api.scrapper.dev/v1/schedules/sch_123/resume \
  -H "X-API-Key: your_api_key"

javascript

await fetch('/api/v1/schedules/sch_123/resume', {
  method: 'POST',
  headers: { 'X-API-Key': 'your_api_key' }
});

Sitemap

Discover URLs from sitemaps and batch-scrape them.

POST

/v1/sitemap

Start a new sitemap discovery crawl.

curl

curl -X POST https://api.scrapper.dev/v1/sitemap \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com/sitemap.xml",
    "max_urls": 500,
    "auto_scrape": true
  }'

javascript

const res = await fetch('/api/v1/sitemap', {
  method: 'POST',
  headers: {
    'X-API-Key': 'your_api_key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    url: 'https://example.com/sitemap.xml',
    max_urls: 500,
    auto_scrape: true
  })
});

GET

/v1/sitemap

List all sitemap jobs.

curl

curl https://api.scrapper.dev/v1/sitemap \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/sitemap', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const { jobs } = await res.json();

GET

/v1/sitemap/:id

Get sitemap job details with discovered URLs.

curl

curl https://api.scrapper.dev/v1/sitemap/sm_123 \
  -H "X-API-Key: your_api_key"

javascript

const res = await fetch('/api/v1/sitemap/sm_123', {
  headers: { 'X-API-Key': 'your_api_key' }
});
const job = await res.json();
console.log(job.urls);