Agentic Workflows Explained: When to Use AI Agents vs Linear Pipelines

TL;DR

• An agentic workflow fixes the goal but lets an AI agent decide the path: it plans, picks tools, checks its own output, and iterates until done.
• Deterministic workflows follow a fixed sequence. Non-agentic AI workflows drop a model into one step of that fixed sequence. Agentic workflows hand runtime control to the agent.
• Use deterministic logic when the path is known and repeatability matters (payments, payroll, alarm clocks). Reach for an agent when the input is open-ended and the plan has to change.
• Real agentic systems need orchestration, planning, tools, memory, state, evaluation, escalation, observability, and cost controls. Skip any of these and the system breaks in production.
• Firecrawl gives agents live web data through a free, keyless MCP server (search, scrape, crawl, interact, agent) so you don't have to build the scraping layer yourself.

An agentic workflow is a system where an AI agent is given a goal and decides at runtime how to reach it: which steps to run, which tools to call, and when the work is done. That's the whole idea. Everything else in this post is about how it differs from a deterministic pipeline, when it's the right choice, and what you actually need to build one that works in production.

Software today runs on three types of workflows:

• Deterministic workflows use no AI whatsoever. If [something] do [something else].
• Non-agentic workflows can be deterministic but we're seeing AI used in them increasingly.

Both deterministic and non-agentic workflows tend to consist of linear steps. An AI agent might parse a document or reshape some data but it has no bearing on the task architecture as a whole. Agentic workflows are flexible. An AI agent has agency. It takes a high-level prompt and converts it into smaller tasks, without additional human input.

• Deterministic workflows: These do not use AI agents. Imagine a traditional data pipeline that scrapes a site and stores the results. This is deterministic.
• Non-agentic workflows: These overlap with deterministic workflows. All deterministic workflows are non-agentic. Non-agentic workflows, however, are not always deterministic. Imagine the same data pipeline we just mentioned. Instead of a hardcoded parser, it might use an AI model to parse site documents and convert them to JSON. The model does not have agency. It does one job and the workflow continues predictably.
• Agentic workflows: This is where our industry is heading. Rather than scraping a site and saving its results, the system itself might decide which sites to scrape and what to do with the results. A useless site might have results discarded. A site with useful data might get stored. A site with potentially malicious data might get added to a blacklist. All of these things are determined by an AI agent. They're not hardcoded before runtime. They're settled by the model during runtime.

An agentic workflow starts with a goal in natural language. The agent breaks that goal into subtasks, picks a tool to run each one, reads the result, and decides what to do next. It loops until it hits a success condition or a stop rule. A human sets the goal and the guardrails, then steps back. The agent does the routing, retrying, and decision-making that a developer would normally hardcode.

That autonomy is starting to reach into work that used to belong entirely to engineers. According to the post below Anthropic actually claims to have succeeded in using Claude to research, test, and train small models entirely on its own. At the moment, this isn't something they're claiming to do in production. It's an experiment they ran.

Successful experiments like this tend to lead to further research and I suspect there could be a production use case in the future.

An agentic workflow is broken into a variety of components. At the top of the system, we often have our orchestrator. It sits between the human operator and the agentic system. Think of it like your assistant manager. The rest of the system is composed of tools, planning, state handling and things requiring human involvement like observation, escalation and cost controls.

• Orchestration: Once we're dealing with multiple agents, we need an orchestration layer. Without one, the agents are off on their own and doing their own thing; like a middle school band class without a conductor.
• Planning: This is central. Agents are usually given high-level tasks: "Summarize [insert site here]." A prompt like that doesn't tell the agent that it needs to scrape the content, read it and then create a distilled summary of the information.
• Tools: Without tooling, AI agents can't actually do anything other than generate text. Tools can be handled via Model Context Protocol (MCP) or through a framework.
• Memory: Agents have limited context. Without basic memory, they can forget what they're doing in the middle of a task. This is where context engineering becomes central: memory lets agents rebuild context when windows get maxed out. It also allows AI agents to fetch additional context when needed. Have you ever had an AI assistant remember useful things from past tasks or chats?
• State management: We need to be able to handle task state. The orchestrator needs to know which tasks are in progress and which ones are completed. If the orchestrator sends a task back to a sub-agent, the sub-agent needs to know to redo the task.
• Evaluation/verification: Agent outputs need to be verified. Without verification, things can quickly get messy. An incomplete job marked as complete can cause problems to ripple throughout an entire system.
• Human-in-the-loop/escalation: Human-in-the-loop was a major part of earlier AI workflows. Today, agentic workflows try to get as close to autonomy as possible. A human should only be brought in if something actually needs judgment. As with traditional software, a good system needs minimal oversight.
• Observability/logging: Observability ties in directly to the escalation component. A human (or AI) reviewing escalated issues needs to be able to see what happened and when. If an agent is taking 20 minutes on a 20 second task, something is wrong. If token usage is spiking, we need to see when the spikes are occurring and what's causing them.
• Cost: This also ties into the components above. Imagine you've got $100 of API credits to use. On a typical task, an agent uses between 1 and 10 cents. When an AI agent goes off track, ideally, it should be turned off as soon as possible and well before your $100 are gone. Latency is also a major factor. Each agent call and each API call creates latency. Wasted calls slow things down and cost us additional tokens.

Agentic workflows have been evolving rapidly into multi-agent systems. It's not uncommon for a system to have one prompt written by a human while the rest are written by AI models in real-time as tasks are processed. Imagine you're actively crawling the web and monitoring the sites of your competitors. Until recently, a standard implementation of this would have been entirely deterministic: if site_text != old_site_text: <add code to alert a human for review>.

Today, that same workflow changes drastically. If site text changes, an AI agent can scrape it immediately and summarize the changes. If you run a grocery store, you need to know when hot dogs are cheaper at a competing store. You don't really need to know when their contact email changed from hello@somewebsite.com to contact@somewebsite.com. If your competition drops prices on all products from an entire brand, it might take a human a couple hours to review and verify. An AI agent can read the prices and tell you immediately which brands changed prices and how big the price drops were.

At first glance, this is a single-agent workflow. If you want scale and redundancy, it's not. When the assembly line was invented, human tasks changed in favor of specialization. At one point on the line, the engine gets dropped into the car. Further down the line, doors and bumpers are added to the car. At the beginning of the line, we have almost nothing.

At the end, we've got a finished product. At the next stage of scaling, companies wanted to build multiple cars at once. The solution wasn't to put two cars on the same line simultaneously. The solution was to have separate lines running concurrently. A supervisor oversees the two lines. The same principle is true in agentic workflows.

At the end of each assembly line, the car needs to be inspected. Afterward, it probably gets a test drive. The same is true for a site monitoring system. One agent converts the price changes into structured data.

You'll likely have a separate agent verify the information. This is standard quality assurance (QA). Once the information is verified, it moves on to the next part of the setup: summarization. If prices didn't change but an email did, the orchestrator should decide to skip right to the report. If a ton of prices changed, we need an explanation. We don't need this explanation from a parsing specialist. We need it from a data analyst; we've got another agent in the picture.

After all of this is finished, our orchestrator marks the task as complete and the report shows up in your inbox.

Multiple assembly lines need a supervisor. Multiple site monitors do too. We don't want to use the exact same assembly line for every car and we don't want to use the exact same parsing system for each website. These portions of the workflow are tuned to fit their specific purpose.

Reach for an agentic workflow when the path is unknown but the goal is clear: competitor monitoring, research, code generation, and open-ended data mining. This is exactly the direction the broader agentic AI trends point toward. Stick with deterministic code when the path is fixed and repeatability matters: payments, payroll, scheduled jobs, anything with a clear rule set. Agents add complexity and reduce predictability, so the payoff has to be real flexibility, not novelty.

If you're a developer in the old sense of the word, you probably remember the "Hello World" program. It's simple and determinism is the entire point. You don't want a program that prints "Hello World" 15 different ways and in 15 different languages. It would just add complexity. The purpose of "Hello World" is to show that your setup is working correctly.

Now let's think of a real-world application that's existed for over a century: alarm clocks. Pretend I set my alarm to go off at 7:00 a.m. It should go off at 7:00 a.m. every time. The clock hits the correct time and the alarm is tripped.

Let's pretend to put an AI agent into this. Instead of tripping the alarm at 7:00 a.m., we now have an agent wired into the clock. It has no clue what time it is. It can check the clock with an MCP call. Because it's software, the tool call is super fast. The AI agent then calls the clock 2,400 times per minute waiting for 7:00 a.m.

We've replaced a deterministic trigger with an intelligent polling architecture. The correct time finally hits and our agent runs out of context. It rebuilds the task from memory and correctly triggers the alarm at 7:01 a.m. The alarm technically goes off, but the process is wasteful, unreliable, and slower than the deterministic version it replaced.

Most agentic workflows depend on fresh web data, and they usually need it on demand. Firecrawl makes this process easy and it's better than built-in web fetching features. Firecrawl gives agents access to the following features.

• Search: Search the web instantly and get real structured results. This allows AI agents to identify data sources and answer basic questions quickly.
• Scrape: Scrape any web page and return it as HTML or Markdown. When you convert a page into Markdown, you can drastically reduce token consumption.
• Interact: Run a real web browser in the cloud and interact with pages. This is best for complex agentic workflows requiring runtime decisions and a stable web browser.
• Crawl: Scrape and extract data from an entire website. This feature is essential for teams amassing data for things like analysis and training.
• Agent: A dedicated research agent powered by Firecrawl's own Spark family of AI models. Input a prompt and let it scour the web for answers. When it's done, you get structured results.

Even if you still write code manually, Firecrawl offers a variety of software development kits (SDKs). If you're building an agentic workflow, it's just a simple MCP connection.

To add the Firecrawl MCP, just point it at the URL below. Firecrawl now offers a free, keyless MCP server.

https://mcp.firecrawl.dev/v2/mcp

You can view the full docs on MCP configuration here.

To add Firecrawl as a Claude Connector, simply search for it using Claude.

Adding Firecrawl to ChatGPT is just as simple as the Claude Connector. Go to the Plugins page and search for Firecrawl.