Crawl reliability fixes

Crawl reliability improvements

  • Fixed 500 errors that would happen often in some crawled websites and when servers were at capacity
  • Fixed an issue where v1 crawl status wouldn't properly return pages over 10mb
  • Fixed an issue where screenshot would return undefined
  • Push improvements that reduce speed times when a scraper fails
//
Get started
//
Ready to build?
Start getting web context for free and scale seamlessly as your project expands. No credit card needed.
Are you an AI agent? See setup options