Introducing Change Tracking: Launch Week III - Day 1

Welcome to Launch Week III, Day 1! Today we’re excited to announce Change Tracking — an enhanced Firecrawl feature that automatically detects and details changes on websites, now available in beta for all users.

Change Tracking allows you to monitor website changes by comparing the current scrapes and crawls to previous versions, clearly indicating if content is new, unchanged, modified, or removed.

Each Change Tracking response includes:

Field	Description
previousScrapeAt	Timestamp of the last scrape (or null if no previous scrape)
changeStatus	new, same, changed, or removed
visibility	visible (found through crawling) or hidden (found via memory)
diff (optional)	Git-style diff of changes (when enabled)
json (optional)	Structured JSON comparison of specific fields (when enabled)

Firecrawl’s Change Tracking feature integrates effortlessly into your existing workflows with two simple request methods—scrape and crawl. You must specify the markdown format in addition to changeTracking:

const scrapeResponse = await app.scrapeUrl('https://firecrawl.dev', {
  formats: ['markdown', 'changeTracking']
});
console.log(scrapeResponse);

Scrape Response:

{
  "url": "https://firecrawl.dev",
  "markdown": "# AI Agents for great customer experiences\n\nChatbots that delight your users...",
  "changeTracking": {
    "previousScrapeAt": "2025-04-10T12:00:00Z",
    "changeStatus": "changed",
    "visibility": "visible"
  }
}

const crawlResponse = await app.crawlUrl('https://firecrawl.dev', {
  scrapeOptions: { formats: ['markdown', 'changeTracking'] }
});
console.log(crawlResponse);

Crawl Response:

{
  "success": true,
  "status": "completed",
  "completed": 2,
  "total": 2,
  "creditsUsed": 2,
  "expiresAt": "2025-04-14T18:44:13.000Z",
  "data": [
    {
      "markdown": "# Turn websites into LLM-ready data\n\nPower your AI apps with clean data crawled from any website...",
      "metadata": {},
      "changeTracking": {
        "previousScrapeAt": "2025-04-10T12:00:00Z",
        "changeStatus": "changed",
        "visibility": "visible"
      }
    },
    {
      "markdown": "## Flexible Pricing\n\nStart for free, then scale as you grow...",
      "metadata": {},
      "changeTracking": {
        "previousScrapeAt": "2025-04-10T12:00:00Z",
        "changeStatus": "changed",
        "visibility": "visible"
      }
    }
  ]
}

Change Tracking supports multiple advanced modes to suit different monitoring needs:

• Git-Diff Mode: Provides detailed, Git-style line-by-line diffs, perfect for content updates and edits.
• JSON Mode: Offers structured comparisons using a custom schema to track specific data changes, ideal for monitoring product details, pricing, or key text changes.

const result = await app.scrapeUrl("http://www.whattimeisit.com", {
  formats: ["markdown", "changeTracking"],
  changeTrackingOptions: {
    modes: ["git-diff", "json"], // Enable specific change tracking modes
    schema: {
      type: "object",
      properties: {
        time: { type: "string" },
      },
    }, // Schema for structured JSON comparison
    prompt: "Get the time", // Optional custom prompt
  },
});

// Access git-diff format changes
if (result.changeTracking.diff) {
  console.log(result.changeTracking.diff.text); // Git-style diff text
  console.log(result.changeTracking.diff.json); // Structured diff data
}

// Access JSON comparison changes
if (result.changeTracking.json) {
  console.log(result.changeTracking.json); // Previous and current values
}

 **April, 13 2025**
 
-**05:55:05 PM**
+**05:58:57 PM**

...

{
  "time": { 
    "previous": "2025-04-13T17:54:32Z", 
    "current": "2025-04-13T17:55:05Z" 
  }
}

When enabled, Firecrawl compares current scrapes against previous versions based on URL, team ID, and markdown format:

• Comparison is resilient to whitespace and content order changes.
• Iframe source URLs are ignored to avoid false positives caused by captchas or antibots.

• URL Consistency: Ensure URLs match exactly for accurate comparisons.
• Scrape Option Consistency: Variations in scrape options can affect consistency.
• Team Scoping: Tracking is scoped per team; initial scrapes always show as new.
• Beta Monitoring: Watch the warning field and handle missing changeTracking objects due to potential database timeouts.

• Basic tracking and Git-diff mode: Free
• JSON mode: 5 credits per page scrape due to additional processing requirements.

Change Tracking is live in beta for all users:

1. Try it now: Add changeTracking to your scrape or crawl formats.
2. Learn more: Read the docs for /scrape and the docs for /crawl.
3. Get help: Join our Discord community or contact help@firecrawl.com.

Ready to track detailed content changes? Sign up for Firecrawl and start today.