Our first Launch Week is over! See the recap 🚀

ChangelogNew

  • Fixes + Improvements (no version bump)

    • Fixed 500 errors that would happen often in some crawled websites and when servers were at capacity
    • Fixed an issue where v1 crawl status wouldn’t properly return pages over 10mb
    • Fixed an issue where screenshot would return undefined
    • Push improvements that reduce speed times when a scraper fails
  • Actions

    Introducing Actions

    Interact with pages before extracting data, unlocking more data from every site!

    Firecrawl now allows you to perform various actions on a web page before scraping its content. This is particularly useful for interacting with dynamic content, navigating through pages, or accessing content that requires user interaction.

    • Version 1.5.x of the Node SDK now supports type-safe Actions.
    • Actions are now available in the REST API and Python SDK (no version bumps required!).

    Here is a python example of how to use actions to navigate to google.com, search for Firecrawl, click on the first result, and take a screenshot.

    from firecrawl import FirecrawlApp
    
    app = FirecrawlApp(api_key="fc-YOUR_API_KEY")
    
    # Scrape a website:
    scrape_result = app.scrape_url('firecrawl.dev', 
        params={
            'formats': ['markdown', 'html'], 
            'actions': [
                {"type": "wait", "milliseconds": 2000},
                {"type": "click", "selector": "textarea[title=\"Search\"]"},
                {"type": "wait", "milliseconds": 2000},
                {"type": "write", "text": "firecrawl"},
                {"type": "wait", "milliseconds": 2000},
                {"type": "press", "key": "ENTER"},
                {"type": "wait", "milliseconds": 3000},
                {"type": "click", "selector": "h3"},
                {"type": "wait", "milliseconds": 3000},
                {"type": "screenshot"}
            ]
        }
    )
    print(scrape_result)
    

    For more examples, check out our API Reference.

  • Firecrawl E2E Type Safe LLM Extract

    Mid-September Updates

    Typesafe LLM Extract

    • E2E Type Safety for LLM Extract in Node SDK version 1.5.x.
    • 10x cheaper in the cloud version. From 50 to 5 credits per extract.
    • Improved speed and reliability.

    Rust SDK v1.0.0

    • Rust SDK v1 is finally here! Check it out here.

    Map Improved Limits

    • Map smart results limits increased from 100 to 1000.

    Faster scrape

    • Scrape speed improved by 200ms-600ms depending on the website.

    Launching changelog

    • For now on, for every new release, we will be creating a changelog entry here.

    Improvements

    • Lots of improvements pushed to the infra and API. For all Mid-September changes, refer to the commits here.
  • September 8, 2024

    Patch Notes (No version bump)

    • Fixed an issue where some of the custom header params were not properly being set in v1 API. You can now pass headers to your requests just fine.
  • Firecrawl V1

    Firecrawl V1 is here! With that we introduce a more reliable and developer friendly API.

    Here is what’s new:

    • Output Formats for /scrape: Choose what formats you want your output in.
    • New /map endpoint: Get most of the URLs of a webpage.
    • Developer friendly API for /crawl/id status.
    • 2x Rate Limits for all plans.
    • Go SDK and Rust SDK.
    • Teams support.
    • API Key Management in the dashboard.
    • onlyMainContent is now default to true.
    • /crawl webhooks and websocket support.

    Learn more about it here.

    Start using v1 right away at https://firecrawl.dev