pip install firecrawl-py python-dotenv

echo "FIRECRAWL_API_KEY='fc-YOUR-KEY-HERE'" >> .env

from firecrawl import Firecrawl
from dotenv import load_dotenv

load_dotenv()
app = Firecrawl()

results = app.search(query="best project management tools", limit=5)

print(f"Found {len(results.web)} web results")
print(f"Search completed successfully")

Found 5 web results
Search completed successfully

# First result details
first_result = results.web[0]
print(f"Title: {first_result.title}")
print(f"URL: {first_result.url}")
print(f"Description: {first_result.description}")
print(f"Category: {first_result.category}")

Title: 25 Best Project Management Software Picked For 2025
URL: https://thedigitalprojectmanager.com/tools/best-project-management-software/
Description: Explore top-rated project management software handpicked by experts to help you manage teams, timelines, and tasks with ease.
Category: None

All 5 results:
1. 25 Best Project Management Software Picked For 2025
   https://thedigitalprojectmanager.com/tools/best-project-management-software/

2. What is the best free project management tool, specifically geared ...
   https://www.reddit.com/r/projectmanagement/comments/1b0lfvi/what_is_the_best_free_project_management_tool/

3. Honest Review of 6 Personal Project Management Tools ... - ICAgile
   https://www.icagile.com/resources/honest-review-of-6-personal-project-management-tools-with-kanban-view

4. Manage your team's work, projects, & tasks online • Asana
   https://asana.com/

5. Project Management Software for Teams - Microsoft
   https://www.microsoft.com/en-us/microsoft-365/planner/project-management

try:
    results = app.search(query="python web scraping", limit=10)
    if results.web:
        print(f"Found {len(results.web)} results")
    else:
        print("No results found")
except Exception as e:
    print(f"Search failed: {e}")

results = app.search(query="", limit=5)

Empty query properly failed: Query cannot be empty

results = app.search(
    query="machine learning frameworks 2025",
    limit=3,
    sources=["web", "news"],
    timeout=30
)

Advanced search found 3 results
1. 8 Best Machine Learning Software To Use in 2025 | Anaconda
2. AI Frameworks: Top Types To Adopt in 2025 - Splunk
3. Uses for ML frameworks like Pytorch/Tensorflow/etc in 2025 - Reddit

# This search finds pages, then scrapes each one automatically
content_results = app.search(
    query="web scraping best practices", 
    limit=2,
    scrape_options={
        "formats": ["markdown", "links"]
    }
)

print(f"Content search found {len(content_results.web)} results")

for i, result in enumerate(content_results.web, 1):
    print(f"\nResult {i}:")
    print(f"Title: {result.metadata.title}")
    print(f"URL: {result.metadata.url}")
    print(f"Content length: {len(result.markdown)} characters")
    print(f"Found {len(result.links)} links in content")

Content search found 2 results

Result 1:
Title: Web Scraping Best Practices and Tools 2025 - ZenRows
URL: https://www.zenrows.com/blog/web-scraping-best-practices
Content length: 25234 characters
Found 51 links in content

Result 2:
Title: 7 Web Scraping Best Practices You Must Be Aware of
URL: https://research.aimultiple.com/web-scraping-best-practices/
Content length: 22384 characters
Found 111 links in content

results = app.search(
    query="python tutorials",
    limit=1,
    scrape_options={
        "formats": ["markdown", "html", "links"]
    }
)

# Each result contains full content in multiple formats
result = results.web[0]
print(f"Markdown: {len(result.markdown)} characters")
print(f"HTML: {len(result.html)} characters") 
print(f"Links: {len(result.links)} found")

Markdown: 33640 characters
HTML: 221019 characters
Links: 302 found

results = app.search(
    query="machine learning research 2025",
    limit=2,
    scrape_options={"formats": ["markdown"]}
)

for result in results.web:
    # Content is already extracted and clean
    content = result.markdown
    word_count = len(content.split())
    
    print(f"Found article: {result.metadata.title}")
    print(f"Content: {word_count} words")

Found article: 2025 Conference
Content: 466 words
Found article: Apple Machine Learning Research at ICML 2025 - Apple Machine Learning Research
Content: 2070 words

# Helper function to display results consistently
def display_results(results, result_type="web", title="Results"):
    result_list = getattr(results, result_type, [])
    print(f"{title}:")
    for result in result_list:
        print(f"- {result.title}")
        print(f"  Category: {getattr(result, 'category', 'N/A')}")
        print(f"  URL: {result.url}")

# Search GitHub for code projects
github_results = app.search(
    query="python web scraping",
    limit=3,
    categories=["github"]
)

# Search research databases for academic papers
research_results = app.search(
    query="machine learning transformers",
    limit=3,
    categories=["research"]
)

display_results(github_results, title="GitHub category results")
print()
display_results(research_results, title="Research category results")

GitHub category results:
- Scrapy, a fast high-level web crawling & scraping ...
  Category: github
  URL: https://github.com/scrapy/scrapy
- oxylabs/Python-Web-Scraping-Tutorial
  Category: github
  URL: https://github.com/oxylabs/Python-Web-Scraping-Tutorial
- ScrapeGraphAI/Scrapegraph-ai: Python scraper based on AI
  Category: github
  URL: https://github.com/ScrapeGraphAI/Scrapegraph-ai

Research category results:
- [2304.10557] An Introduction to Transformers
  Category: research
  URL: https://arxiv.org/abs/2304.10557
- A Comprehensive Survey on Applications of Transformers ...
  Category: research
  URL: https://arxiv.org/abs/2306.07303
- Multimodal Learning With Transformers: A Survey
  Category: research
  URL: https://pubmed.ncbi.nlm.nih.gov/37167049/

# Helper function for source comparison
def compare_sources(query, limit=2):
    web_results = app.search(query=query, limit=limit, sources=["web"])
    news_results = app.search(query=query, limit=limit, sources=["news"])
    
    print("Web source results:")
    for result in web_results.web:
        domain = result.url.split('/')[2] if '/' in result.url else result.url
        print(f"- {result.title}")
        print(f"  Domain: {domain}")
    
    print("\nNews source results:")
    if hasattr(news_results, 'news') and news_results.news:
        for result in news_results.news:
            print(f"- {result.title}")
            print(f"  Date: {getattr(result, 'date', 'N/A')}")

compare_sources("AI developments 2025")

Web source results:
- The 2025 AI Index Report | Stanford HAI
  Domain: hai.stanford.edu
- The Latest AI News and AI Breakthroughs that Matter Most: 2025
  Domain: www.crescendo.ai

News source results:
- Microsoft's Unprecedented 2025: Cloud and AI Fuel Record Earnings, Igniting Shareholder Optimism
  Date: 12 hours ago
- 12 Graphs That Explain the State of AI in 2025
  Date: Apr 7, 2025

# Basic image search
basic_images = app.search(
    query="mountain landscape",
    limit=3,
    sources=["images"]
)

print("Basic image search results:")
if hasattr(basic_images, 'images') and basic_images.images:
    print(f"Found {len(basic_images.images)} images")
    for i, img in enumerate(basic_images.images, 1):
        print(f"{i}. {img.title}")
        print(f"   Source page: {img.url}")
        print(f"   Position: {img.position}")

Basic image search results:
Found 3 images
1. Free Sunset Mountain Landscape Image | Download at StockCake
   Source page: https://stockcake.com/i/sunset-mountain-landscape_1219985_159231
   Position: 1

2. Watercolor Mountain Landscape Tutorial: Step-by-Step Guide
   Source page: https://www.esperoart.com/mountain-landscape-watercolor-tutorial/
   Position: 2

3. Mountain Landscape with Forests and a Stream · Free Stock Photo
   Source page: https://www.pexels.com/photo/mountain-landscape-with-forests-and-a-stream-18682254/
   Position: 3

def search_images_by_resolution():
    # Search for Full HD images
    hd_images = app.search(
        query="sunset landscape imagesize:1920x1080",
        limit=3,
        sources=["images"]
    )

    print("HD image search results:")
    if hasattr(hd_images, 'images') and hd_images.images:
        for i, img in enumerate(hd_images.images, 1):
            print(f"{i}. {img.title}")
            print(f"   Source: {img.url}")

    # Search for 4K images
    uhd_results = app.search(
        query="nature wallpaper imagesize:3840x2160",
        limit=2,
        sources=["images"]
    )

    print("\n4K image search results:")
    if hasattr(uhd_results, 'images') and uhd_results.images:
        for img in uhd_results.images:
            print(f"- {img.title}")
            print(f"  Source: {img.url}")

search_images_by_resolution()

HD image search results:
1. Stunning sunset landscape: Mediterranean rocks with olive trees in the sea.  Photorealistic motion background. 3D rendering.
   Source: https://www.storyblocks.com/video/stock/photorealistic-motion-background-mediteranean-rocks-in-the-sea-with-olive-trees-growing-on-them-beautiful-sunset-landscape-scene-3d-rendering-bymitreh8kaz2db71

2. Sunset in a hilly rural landscape on the edge of a forest, Timelapse video
   Source: https://www.storyblocks.com/video/stock/sunset-in-a-hilly-rural-landscape-on-the-edge-of-a-forest-timelapse-video-351768226

4K image search results:
- 50 Nature Wallpapers (All 4K, No watermarks) - Album on Imgur
  Source: https://in.pinterest.com/pin/1002543567023869780/
- My top 10 of all time - Nature and Art : r/wallpapers
  Source: https://www.reddit.com/r/wallpapers/comments/1g1cymc/my_top_10_of_all_time_nature_and_art/

def compare_content_freshness(base_query):
    recent_results = app.search(
        query=f"{base_query} trends",
        limit=3,
        tbs="qdr:m"  # Past month
    )

    all_time_results = app.search(
        query=f"{base_query} fundamentals",
        limit=3
    )

    print("Recent results (past month):")
    for result in recent_results.web:
        print(f"- {result.title}")

    print("\nAll-time results (fundamentals):")
    for result in all_time_results.web:
        print(f"- {result.title}")

compare_content_freshness("machine learning")

Recent results (past month):
- Top 13 Machine Learning Trends CTOs Need to Know in 2025
- The Evolution of AI and ML: Trends, Impact, and Future Insights
- The Ultimate List of Machine Learning Statistics for 2025 - Itransition

All-time results (fundamentals):
- Introduction to Machine Learning Concepts - Training - Microsoft Learn
- Machine Learning Crash Course - Google for Developers
- Machine Learning Tutorial - GeeksforGeeks

multi_category_results = app.search(
    query="neural networks",
    limit=4,
    categories=["github", "research"]
)

print("Combined GitHub + Research results:")
for result in multi_category_results.web:
    print(f"- {result.title}")
    print(f"  Category: {getattr(result, 'category', 'N/A')}")
    print(f"  URL: {result.url}")

Combined GitHub + Research results:
- Neural Networks | Journal | ScienceDirect.com by Elsevier
  Category: research
  URL: https://www.sciencedirect.com/journal/neural-networks
- Neural Network - an overview | ScienceDirect Topics
  Category: research
  URL: https://www.sciencedirect.com/topics/social-sciences/neural-network
- Neural Network - an overview
  Category: research
  URL: https://www.sciencedirect.com/topics/computer-science/neural-network
- Supervised learning in DNA neural networks
  Category: research
  URL: https://www.nature.com/articles/s41586-025-09479-w

echo "FIRECRAWL_API_KEY='fc-YOUR-FIRECRAWL-KEY'" >> .env
echo "OPENAI_API_KEY='sk-YOUR-OPENAI-KEY'" >> .env

# Install uv if you don't have it
curl -LsSf https://astral.sh/uv/install.sh | sh

source ~/.bashrc  # or ~/.zshrc

# uv expects to work in a Python project directory with a pyproject.toml file
uv init

# Install project dependencies
uv add firecrawl-py python-dotenv langchain-openai langgraph

pip install firecrawl-py python-dotenv langchain-openai langgraph

#!/usr/bin/env python3
"""Research Assistant Agent using LangGraph and Firecrawl Search API"""

import os
from dotenv import load_dotenv
from firecrawl import Firecrawl
from langchain_openai import ChatOpenAI
from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import MemorySaver

# Load environment variables
load_dotenv()
firecrawl = Firecrawl()

def general_web_search(query: str, limit: int = 3) -> str:
    """Search the web for general information and extract full content."""
    try:
        results = firecrawl.search(
            query=query,
            limit=limit,
            sources=["web"],
            scrape_options={"formats": ["markdown"]}
        )
        
        if not results.web:
            return f"No web results found for: {query}"
        
        formatted_results = f"Web Search Results for '{query}':\n\n"
        for i, result in enumerate(results.web, 1):
            title = result.metadata.title if hasattr(result, 'metadata') else result.title
            url = result.metadata.url if hasattr(result, 'metadata') else result.url
            content_preview = result.markdown[:300] + "..." if len(result.markdown) > 300 else result.markdown
            
            formatted_results += f"{i}. {title}\n"
            formatted_results += f"   URL: {url}\n"
            formatted_results += f"   Content: {content_preview}\n\n"
        
        return formatted_results
        
    except Exception as e:
        return f"Error in web search: {str(e)}"

def academic_research_search(topic: str, limit: int = 3) -> str:
    """Search for academic research papers and scholarly content."""
    try:
        results = firecrawl.search(
            query=topic,
            limit=limit,
            categories=["research"],
            scrape_options={"formats": ["markdown"]}
        )
        
        if not results.web:
            return f"No research papers found for: {topic}"
        
        formatted_results = f"Academic Research Results for '{topic}':\n\n"
        for i, result in enumerate(results.web, 1):
            title = result.metadata.title if hasattr(result, 'metadata') else result.title
            url = result.metadata.url if hasattr(result, 'metadata') else result.url
            category = getattr(result, 'category', 'research')
            
            formatted_results += f"{i}. {title}\n"
            formatted_results += f"   Category: {category}\n"
            formatted_results += f"   URL: {url}\n\n"
        
        return formatted_results
        
    except Exception as e:
        return f"Error in research search: {str(e)}"

def code_search(technology: str, limit: int = 3) -> str:
    """Search GitHub repositories for code examples and projects."""
    try:
        results = firecrawl.search(
            query=technology,
            limit=limit,
            categories=["github"]
        )
        
        if not results.web:
            return f"No GitHub repositories found for: {technology}"
        
        formatted_results = f"GitHub Code Search Results for '{technology}':\n\n"
        for i, result in enumerate(results.web, 1):
            formatted_results += f"{i}. {result.title}\n"
            formatted_results += f"   URL: {result.url}\n"
            formatted_results += f"   Description: {result.description or 'No description available'}\n\n"
        
        return formatted_results
        
    except Exception as e:
        return f"Error in code search: {str(e)}"

def news_search(topic: str, time_filter: str = "qdr:w", limit: int = 3) -> str:
    """Search for recent news articles on a topic."""
    try:
        results = firecrawl.search(
            query=topic,
            limit=limit,
            sources=["news"],
            tbs=time_filter
        )
        
        news_results = []
        if hasattr(results, 'news') and results.news:
            news_results = results.news
        elif hasattr(results, 'web') and results.web:
            news_results = results.web
        
        if not news_results:
            return f"No recent news found for: {topic}"
        
        time_desc = {"qdr:d": "past day", "qdr:w": "past week", "qdr:m": "past month"}.get(time_filter, "recent")
        formatted_results = f"News Search Results for '{topic}' ({time_desc}):\n\n"
        
        for i, result in enumerate(news_results, 1):
            formatted_results += f"{i}. {result.title}\n"
            formatted_results += f"   URL: {result.url}\n"
            formatted_results += f"   Date: {getattr(result, 'date', 'Date not available')}\n\n"
        
        return formatted_results
        
    except Exception as e:
        return f"Error in news search: {str(e)}"

def create_research_agent():
    """Create the research assistant agent with search tools."""
    
    model = ChatOpenAI(model="gpt-5", temperature=0)
    tools = [general_web_search, academic_research_search, code_search, news_search]
    
    agent = create_react_agent(
        model=model,
        tools=tools,
        checkpointer=MemorySaver()
    )
    
    return agent

def main():
    """Main function to run the research agent."""
    print("🔍 Research Assistant Agent")
    print("=" * 50)
    print("I can help you search for:")
    print("• General web information with full content")
    print("• Academic research papers")
    print("• GitHub code repositories") 
    print("• Recent news articles")
    print("\nType 'quit' to exit")
    print("=" * 50)
    
    agent = create_research_agent()
    
    while True:
        user_input = input("\n💬 What would you like to research? ")
        
        if user_input.lower() in ['quit', 'exit', 'q']:
            print("👋 Goodbye!")
            break
            
        if not user_input.strip():
            continue
            
        try:
            print("\n🤖 Searching...")
            
            system_message = """You are a Research Assistant Agent that helps users find information on any topic.

You have access to four specialized search tools:
1. general_web_search - For web search with full content extraction
2. academic_research_search - For finding academic papers and scholarly content  
3. code_search - For finding GitHub repositories and code examples
4. news_search - For finding recent news articles

Choose the appropriate tool(s) based on the user's request. You can use multiple tools for extensive research."""
          
            response = agent.invoke(
                {"messages": [("system", system_message), ("user", user_input)]},
                config={"configurable": {"thread_id": "research_session"}}
            )
            
            final_message = response["messages"][-1]
            print(f"\n📋 Research Results:\n{final_message.content}")
            
        except Exception as e:
            print(f"\n❌ Error: {str(e)}")

if __name__ == "__main__":
    main()

python research_agent.py

🔍 Research Assistant Agent
==================================================
I can help you search for:
• General web information with full content
• Academic research papers
• GitHub code repositories
• Recent news articles

Type 'quit' to exit
==================================================

💬 What would you like to research?

🤖 Searching...

📋 Research Results:
Here are some recent machine learning frameworks for computer vision found on GitHub:

1. OpenCV: An open-source library providing computer vision and machine learning 
   algorithms for image and video analysis. It also includes Detectron2, a 
   high-performance object detection framework.

2. 500 AI Machine learning Deep learning Computer vision NLP Projects with code: 
   This repository contains a list of 500+ AI projects, including machine learning 
   and computer vision, with code.

3. Lightly: A computer vision framework for self-supervised learning. The repository 
   also includes Learnergy, energy-based machine learning models built upon PyTorch.

🤖 Searching...

📋 Research Results:
Here are some academic research papers on neural networks:

1. Neural Networks Journal | ScienceDirect: The official journal for neural 
   network research with peer-reviewed articles.

2. Neural Network - an overview | ScienceDirect Topics: Comprehensive overview 
   of neural networks in the context of social sciences.

3. Overview of artificial neural networks - PubMed: Medical and biological 
   applications of artificial neural networks.

🤖 Searching...

📋 Research Results:
Here are some Python web scraping libraries and projects on GitHub:

1. Web Scraping With Python: Tutorial using requests and Beautiful Soup libraries.

2. Python-Web-Scraping-Tutorial by oxylabs: Comprehensive tutorial from simple 
   examples to complex tasks.

3. Scrapling by D4Vinci: Undetectable, powerful, flexible Python library for 
   web scraping.

4. Scrapegraph-ai: Web scraping python library that uses LLM and direct graph 
   logic to create scraping pipelines.

async def stream_agent_response(agent, query):
    """Stream agent responses for real-time feedback."""
    async for chunk in agent.astream(
        {"messages": [("user", query)]},
        config={"configurable": {"thread_id": "user_session"}}
    ):
        if "messages" in chunk:
            yield chunk["messages"][-1].content