← Web Search LLM Leaderboard

DeepSeek V4.1 Flash (1017)

DeepSeek · open weights · data as of

DeepSeek V4.1 Flash (1017) ranks 5 of 10 on the Web Search LLM Leaderboard, across 4,703 questions from BrowseComp, DeepSearchQA, HLE and Firecrawl Search Bench. It shares that rank with Mercury 3.

Score with Firecrawl Alexandria68.9
+38.0 points of lift, from 30.9 without
050100
Without With Firecrawl Alexandria

DeepSeek V4.1 Flash (1017) on each question set

Points out of 100. Rank is among the 10 models on that set, by score with Firecrawl Alexandria; vs median is the gap to the median model there. Firecrawl Search Bench is our own question set; its questions are not published.

Question setWithoutWithLiftRankvs median
BrowseComp19.658.5+38.9#5+2.0
DeepSearchQA44.180.1+36.0#6−0.1
HLE40.643.8+3.1#7−4.2
Firecrawl Search Bench19.293.2+74.0#8−0.8

Cost and speed

$289 per 1,000 questions, model tokens plus Firecrawl credits: 1.3× the median model, 7th cheapest of 10.

Time per question
368 s
average, 7th fastest of 10
Median time
186 s
set-weighted, half take longer
90th percentile time
1019 s
set-weighted, 1 in 10 take longer
Failed
0.7%
of questions, scored 0

DeepSeek V4.1 Flash (1017) vs its closest rivals

  • Mercury 3 scores 68.7, within the margin of error. It is 1.2× cheaper and takes 214 s per question on average.
  • MiniMax M3.1 Flash scores 64.8, clearly behind. It is 1.4× cheaper and takes 407 s per question on average.
  • Claude Haiku 5.5 scores 61.9, clearly behind. It is 1.7× cheaper and takes 156 s per question on average.
//
Get started
//
Ready to build?
Start getting web context for free and scale seamlessly as your project expands. No credit card needed.
Start for freeSee our plans
Are you an AI agent? See setup options