← Web Search LLM Leaderboard
Nemotron 3.5 Lightning
NVIDIA · open weights · data as of
Nemotron 3.5 Lightning ranks 10 of 10 on the Web Search LLM Leaderboard, across 4,703 questions from BrowseComp, DeepSearchQA, HLE and Firecrawl Search Bench.
Score with Firecrawl Alexandria44.0
+32.8 points of lift, from 11.2 without050100
Without With Firecrawl Alexandria
Nemotron 3.5 Lightning on each question set
Points out of 100. Rank is among the 10 models on that set, by score with Firecrawl Alexandria; vs median is the gap to the median model there. Firecrawl Search Bench is our own question set; its questions are not published.
| Question set | Without | WithWith Firecrawl Alexandria | Lift | Rank | vs median |
|---|---|---|---|---|---|
| BrowseComp | 1.2 | 17.2 | +16.0 | #10 | −39.2 |
| DeepSearchQA | 14.6 | 49.9 | +35.4 | #10 | −30.3 |
| HLE | 14.2 | 20.3 | +6.1 | #10 | −27.7 |
| Firecrawl Search Bench | 14.8 | 88.5 | +73.7 | #10 | −5.4 |
Cost and speed
$89 per 1,000 questions, model tokens plus Firecrawl credits: 0.4× the median model, cheapest of 10.
- Time per question
- 111 s
- average, fastest of 10
- Median time
- 78 s
- set-weighted, half take longer
- 90th percentile time
- 253 s
- set-weighted, 1 in 10 take longer
- Failed
- <0.1%
- of questions, scored 0
Nemotron 3.5 Lightning vs its closest rivals
- GLM 5.3 Flash scores 61.9, clearly ahead. It costs 2.0× as much and takes 755 s per question on average.
- Claude Haiku 5.5 scores 61.9, clearly ahead. It costs 1.9× as much and takes 156 s per question on average.
- MiniMax M3.1 Flash scores 64.8, clearly ahead. It costs 2.3× as much and takes 407 s per question on average.
- Ranks count only the models clearly ahead, so models too close to call share a rank. How we tested →
- What score and lift mean →What is Firecrawl Alexandria? →Firecrawl search docs →
//
Get started
//
Ready to build?
Start getting web context for free and scale seamlessly as your project expands. No credit card needed.
Are you an AI agent? See setup options