← Web Search LLM Leaderboard
Mercury 3
Inception · closed weights · data as of
Mercury 3 ranks 5 of 10 on the Web Search LLM Leaderboard, across 4,703 questions from BrowseComp, DeepSearchQA, HLE and Firecrawl Search Bench. It shares that rank with DeepSeek V4.1 Flash (1017).
Score with Firecrawl Alexandria68.7
+42.4 points of lift, from 26.4 without050100
Without With Firecrawl Alexandria
Mercury 3 on each question set
Points out of 100. Rank is among the 10 models on that set, by score with Firecrawl Alexandria; vs median is the gap to the median model there. Firecrawl Search Bench is our own question set; its questions are not published.
| Question set | Without | WithWith Firecrawl Alexandria | Lift | Rank | vs median |
|---|---|---|---|---|---|
| BrowseComp | 12.2 | 54.4 | +42.3 | #6 | −2.0 |
| DeepSearchQA | 41.7 | 80.4 | +38.7 | #5 | +0.1 |
| HLE | 36.4 | 46.2 | +9.7 | #6 | −1.9 |
| Firecrawl Search Bench | 15.2 | 93.9 | +78.8 | #5 | 0.0 |
Cost and speed
$242 per 1,000 questions, model tokens plus Firecrawl credits: 1.1× the median model, 6th cheapest of 10.
- Time per question
- 214 s
- average, 5th fastest of 10
- Median time
- 107 s
- set-weighted, half take longer
- 90th percentile time
- 576 s
- set-weighted, 1 in 10 take longer
- Failed
- 0.1%
- of questions, scored 0
Mercury 3 vs its closest rivals
- DeepSeek V4.1 Flash (1017) scores 68.9, within the margin of error. It costs 1.2× as much and takes 368 s per question on average.
- MiniMax M3.1 Flash scores 64.8, clearly behind. It is 1.2× cheaper and takes 407 s per question on average.
- Claude Haiku 5.5 scores 61.9, clearly behind. It is 1.5× cheaper and takes 156 s per question on average.
- * Mercury 3 sometimes declines to answer; each refusal counts as a wrong answer.
- Ranks count only the models clearly ahead, so models too close to call share a rank. How we tested →
- What score and lift mean →What is Firecrawl Alexandria? →Firecrawl search docs →
//
Get started
//
Ready to build?
Start getting web context for free and scale seamlessly as your project expands. No credit card needed.
Are you an AI agent? See setup options