Track down the post where a writer first made a point, list everything a newsletter has published and read the free posts in full, all without scrolling an archive page by hand. Once the Firecrawl connector is switched on, Claude is able to search a Substack archive using Firecrawl Alexandria, working from the archive and search that Substack provides for every publication.
Best for
- Readers looking for the post in which a newsletter author first laid out an idea.
- Researchers collecting every post a writer has published on a single subject.
- Writers and editors reviewing what a newsletter has already covered before they pitch or publish.
Connector and data providers
Connector: the Firecrawl connector. Via this connector, Claude calls the Substack tools in Firecrawl Alexandria at 5 credits per lookup, billed to the Firecrawl team you chose at setup.
Data providers in Firecrawl Alexandria
| Provider | Tool | What you get |
|---|---|---|
| Substack | Archive | Up to 50 posts per page, ordered newest, most popular or pinned. Each entry carries the title and subtitle, URL, publish time, access level, word count, reactions, comments, restacks and author. An offset reaches older pages. |
| Substack | Search | Substack's built-in archive search, scoped to one publication, returning entries in the same shape as the archive. |
| Substack | Post | A single post fetched by URL, including its body. Free posts arrive whole; paid ones arrive as the public preview, with a paywall flag and the proportion of words you can see. |
Explore Firecrawl Alexandria data providers across categories
Starter prompt
Change Noahpinion and tariffs to the newsletter and subject you're interested in:
Use Firecrawl to search the Noahpinion Substack archive in Firecrawl
Alexandria for posts about tariffs. Tell me each tool's price before you
run it.
I want:
- All matching posts, with title, date, link and free or paid status.
- The three that drew the most discussion, judged by comments and
reactions.
- A brief summary of the most recent free post, based on its full text.
Label paid posts as previews, and quote only text the tool returned.Before you start
- Add the Firecrawl connector. Go to its directory listing and press Connect. You'll land on Firecrawl in your browser to sign in or register for free, pick a team and authorize Claude. The connector needs no API key or MCP configuration, and Firecrawl in Claude explains each step.
- Switch Firecrawl on per conversation. It's listed under Connectors in the + menu.
- Have the newsletter's slug handy. For a substack.com publication, use the part before the domain (garymarcus for garymarcus.substack.com). Among custom-domain newsletters, Noahpinion, Lenny's Newsletter and Astral Codex Ten are the three supported right now.
Limitations of using Claude to search a Substack archive
A long-running Substack mixes free posts, paywalled posts and posts search engines never indexed. Five problems follow when Claude tries to search one.
- Search surfaces a sample. Web search grounds Claude's answers in live pages with citations, yet Anthropic notes that "the quality of Claude's responses depends on the underlying sources it references" (Anthropic). When search returns three posts on a topic, it says nothing about the dozen it missed.
- Back issues may be paywalled. Writers can set Substack to paywall their archive automatically, which shuts free subscribers and newcomers out of older free posts, and anything paywalled manually stays locked (Substack). A non-paying reader only gets part of them.
- Private publications are invisible to search engines. Substack says a private newsletter's Welcome page may be indexed while its posts won't be (Substack). A search can confirm the newsletter exists and still show none of its writing.
- Recent posts sit past the model's cutoff. Claude models have knowledge cutoffs, and Anthropic says a model's answers about recent events can be inaccurate (Anthropic). On Team and Enterprise plans an Owner also has to turn web search on for the workspace before Claude can look anything up (Anthropic).
- Posts and quotes that were never written. Anthropic's help center admits Claude sometimes displays quotes that look credible yet have no factual basis, and tells users to check what Claude cites (Anthropic). A plausible headline from a writer you read is easy to fabricate, so click through before you repeat a line.
How it works
Asking Claude about a newsletter draws on four pieces:
- Firecrawl Alexandria: structured data tools an agent calls directly. Firecrawl pays the 100+ official providers in it whenever their data is used, and public sources such as Substack sit alongside them, as do the Web, Research, Developer and Government indexes Firecrawl builds. Firecrawl had a blind AI judge score 845 tasks where only the toolset changed, and agents equipped with Firecrawl Alexandria came out 21% ahead on answer quality over agents working from the built-in web tools alone.
- The archive the publication serves: Substack tools read each newsletter's own archive and search, so they reach posts that web search might never rank. Claude steps back through older posts 50 per call, and when we paged it through the API on Oct 9, 2026, Noahpinion's archive went back to 2022.
- Free and paid, clearly marked: each post entry states its access level. Opening a paid post gives only the preview an anonymous reader would see, flagged as paywalled with the visible fraction of words.
- Read-only, with prices shown first: Claude displays the credit cost before calling a tool, and every tool only reads public pages. No post is published, liked or subscribed to on your behalf.
Searching a Substack archive in Claude with and without Firecrawl Alexandria
| The open web | The Firecrawl connector | |
|---|---|---|
| Posts you'll see | The few a search engine chooses to rank | The newsletter's own archive, paged back as far as it goes |
| Topic search | Only as good as the indexing | Substack's search, run against that one publication |
| Free or paid | Usually unknown until you open the link | Shown for every post, with a paywall flag |
| Opening a post | A fetch that can fail or halt at the paywall | Complete text for free posts; for paid ones, the public preview and how much of it is visible |
| Popularity | Seldom part of search results | Reactions, comments and restacks on each post, plus a most-popular ordering |
| Source | No way to tell what was skipped | Substack's public archive, a source in Firecrawl Alexandria, a catalog of 100+ providers that stretches from media to company and government data. Explore the catalog |
| New posts | You revisit the page yourself | You can create a Firecrawl Monitor yourself in Firecrawl that checks the archive page on a schedule and sends email or Slack alerts whenever it updates |
Build your own Substack archive workflow
Fit the prompt to your goal. For a researcher tracing how an argument developed:
With Firecrawl, search the garymarcus Substack for posts on scaling. Give
me each match's date, title and link, then read the two oldest free posts
in full and explain how the argument began.For a writer evaluating a newsletter before pitching:
Use Firecrawl to list the 20 most popular Lenny's Newsletter posts with
reactions, comments, word count and free or paid status. What subjects
recur most?Tell Claude which publication, whether you want search hits, the latest posts or the most popular ones, and which posts it should open and read.
Take it further
Get up to date on a newsletter
Use Firecrawl to list the 10 latest Astral Codex Ten posts with dates and
free or paid status, then summarize the most recent free one.Dig through the back catalog
With Firecrawl, page through the Noahpinion archive and list every 2023
post with its title and link.See what the paywall covers
Use Firecrawl to open
https://www.noahpinion.blog/p/the-second-trump-presidency-is-a and tell me
what share of the post a non-subscriber can read.Compare prices before running anything
With Firecrawl, which tools can search or read a Substack newsletter? List
each with its price and don't run any yet.What it costs
There's no charge for browsing Firecrawl Alexandria or checking prices. Credits are spent only when a Substack tool runs, and Firecrawl pricing shows how many each plan includes.
| Substack tool | Returns | Credits per call |
|---|---|---|
| Archive | Up to 50 posts per page, newest, top or pinned first | 5 |
| Search | Matching posts within one publication | 5 |
| Post | The text of one post, whole if free, preview if paid | 5 |
Going back 300 posts means six archive calls, which comes to 30 credits. Claude notes the tool and its credit cost after each answer.
[ Substack archives in Claude ]
Connect Firecrawl and have Claude search any newsletter's back catalog. Finding the right tool is free.
Related use cases
- How to search podcast transcripts in Claude: plenty of newsletter authors run podcasts too, and this surfaces what they said in their episodes.
- How to get real-time stock news in Claude: look up the market headlines a finance newsletter is responding to.
- Nasdaq.com: the full body and precise publication time of each story in its news feeds, handy for checking the market news behind a post.
Coverage limits
- Paid posts stop at the preview. You get what a logged-out reader sees; one paid Noahpinion post we fetched through the API on Oct 9, 2026 exposed 598 of its 2,933 words.
- Which addresses work. Publications on substack.com work by slug. So far the only custom domains are Noahpinion, Astral Codex Ten and Lenny's Newsletter.
- No filtering by date. Sorting is by newest, most popular or pinned, with paging by offset, so reaching a particular year means paging back until you get there.