How does web search work in OpenClaw?
OpenClaw's web pipeline runs two tools in sequence: web_search sends a query to a configured provider (Brave, Gemini, or Perplexity) and returns URLs and snippets, then web_fetch makes a plain HTTP GET request to read each URL as markdown. The handoff breaks on modern sites because web_fetch does not execute JavaScript: pages that render content client-side return empty or incomplete content, and many sites return 403 errors to plain HTTP requests.
| Factor | web_search | web_fetch |
|---|---|---|
| What it does | Sends query to search provider, returns URLs and snippets | Fetches a specific URL and extracts readable content |
| JavaScript rendering | Not applicable | No |
| Output | Title, URL, snippet per result | Markdown or plain text |
| Configurable provider | Brave (default), Gemini, Perplexity | N/A (HTTP GET) |
| Firecrawl integration | Replaced by Firecrawl CLI skill | Fallback extractor via API key config |
Use Brave for tasks where snippets are enough to rank sources. For full page content, the Firecrawl CLI skill's firecrawl search returns results and scraped markdown in one call, skipping the web_fetch round-trip. Add your Firecrawl API key to the web_fetch config to fix failed fetches on JS-heavy or bot-protected sites without changing how search works.
The Firecrawl search endpoint returns full scraped content per result, making it a direct replacement for the web_search + web_fetch pipeline in OpenClaw research workflows. See the OpenClaw web search guide for setup and configuration details.
data from the web