Deep Research

Deep Research agent for information collection and report generation.

Deep Research Agent is proficient in gathering and synthesizing information from multiple sources to generate comprehensive reports that address complex user queries. By integrating key capabilities including task decomposition & expansion, deep search, self reflection, and report generation into flexible and callable tools, Deep Research Agent can generate detailed, well-organized reports that help users gain deeper insights towards the queried task.

---

Quick Start

1. Prerequisites

   • Python 3.10 or higher
   • Node.js and npm (for the MCP server)
   • DashScope API key from Alibaba Cloud
   • Tavily search API key from Tavily
2. Installation

   cd agentscope
   pip install -r requirements.txt
   # Installing Tavily tool for web search
   pip install tavily
   
3. Configuration

   Set up the following necessary API keys:

   export DASHSCOPE_API_KEY="your_dashscope_api_key_here"
   export TAVILY_API_KEY="your_tavily_api_key_here"
   export AGENT_OPERATION_DIR="your_own_direction_here"
   

   Before running the Deep Research Agent, you can test whether you can start the Tavily MCP server.

   npx -y tavily-mcp@latest
   
4. Connect to Web Search MCP Client

   The DeepResearch Agent only supports web search through the Tavily MCP client currently. To use this feature, you need to start the MCP server locally and establish a connection to it.

   from agentscope.mcp import StdIOStatefulClient
   
   tavily_search_client= StdIOStatefulClient(
       name="tavily_mcp",
       command="npx",
       args=["-y", "tavily-mcp@latest"],
       env={"TAVILY_API_KEY": os.getenv("TAVILY_API_KEY", "")},
   )
   await tavily_search_client.connect()
   
5. Usage

   You can start running the Deep Research agent in your terminal with the following command:

   cd agentscope/examples/agent/deep_research_agent
   python main.py
   

   We have already provided a query case in main.py for your testing. You can test on your own data by modifying the query:

   if __name__ == "__main__":
       query = (
           "If Eliud Kipchoge could maintain his record-making "
           "marathon pace indefinitely, how many thousand hours "
           "would it take him to run the distance between the "
           "Earth and the Moon its closest approach? Please use "
           "the minimum perigee value on the Wikipedia page for "
           "the Moon when carrying out your calculation. Round "
           "your result to the nearest 1000 hours and do not use "
           "any comma separators if necessary."
       )
       try:
           asyncio.run(main(query))
       except Exception as e:
           logger.exception(e)
   

   If you want to have multi-turn conversations with the Deep Research Agent, you can add the following code in main.py:

   from agentscope.agent import UserAgent
   user = UserAgent("User")
   user_msg = None
   msg = []
   while True:
       user_msg = await user(user_msg)
       if user_msg.get_text_content() == "exit":
           break
       msg.append(user_msg)
       assistant_msg = await agent(msg)
       msg.append(assistant_msg)
   

   The example is built with DashScope chat model. If you want to change the model, ensure you also update the formatter accordingly. The correspondence between built-in models and formatters is listed in the Provider Reference.

---

Key Features

Task Decomposition & Expansion

• Task Decomposition: The system implements a depth-first search strategy that dynamically decomposes tasks during the research process, transforming the workflow from linear to tree-structured. Decomposition can be triggered at any point during agent operation, enabling autonomous knowledge exploration, with automatic backtracking to parent tasks upon subtask completion to maintain logical consistency.
• Stack-based Mechanism: Employs a stack structure to explicitly manage task sequences and context, ensuring logical consistency and stable backtracking across complex research paths.
• Task Expansion: Enhances insight depth by analyzing tasks through eight professional dimensions, guiding the agent to explore multifaceted knowledge beyond the initial query.

async def decompose_and_expand_subtask(self) -> ToolResponse:
    """Identify the knowledge gaps of the current subtask and generate a
    working plan by subtask decomposition. The working plan includes
    necessary steps for task completion and expanded steps.

    Returns:
        ToolResponse:
            The knowledge gaps and working plan of the current subtask
            in JSON format.
    """

Deep Search

• Breadth & Depth Integration: Combines wide-range web searching via multiple queries with high-fidelity extraction of high-value web content for granular analysis.
• Recursive Information Filling: Automatically converts identified information gaps into new sub-tasks, triggering further decomposition to ensure comprehensive coverage of the research topic.

async def _follow_up(
    self,
    search_results: list | str,
    tool_call: ToolUseBlock,
) -> ToolResponse:
    """Read the website more intensively to mine more information for
    the task. And generate a follow-up subtask if necessary to perform
    deep search.
    """

Self Reflection

• Low-level Reflection: Low-level reflection involves corrective measures for issues arising from tool errors, incorrect parameter usage, or ineffective sub-task completion. These are resolved by adjusting decision-making in subsequent steps of the ReAct process.
• High-level Reflection: High-level reflection addresses persistent failures that resist simple corrections, often indicating unanticipated practical challenges in the initial planning. In such cases, the agent may rephrase current steps if there is a misunderstanding of sub-task objectives or if they are unachievable in their current forms.

async def reflect_failure(self) -> ToolResponse:
    """Reflect on the failure of the action and determine to rephrase
    the plan or deeper decompose the current step.

    Returns:
        ToolResponse:
            The reflection about plan rephrasing and subtask decomposition.
    """

Report Generation

• Concurrent Drafting & Citation: Maintains a real-time intermediate document with full traceability, recording findings and citations as research progresses to prevent knowledge loss.
• Synthesis-based Finalization: Shifts the generation focus from “writing from scratch” to “summarizing and polishing” intermediate records, significantly improving the report’s information density, readability, and structural logic.

async def summarize_intermediate_results(self) -> ToolResponse:
    """Summarize the intermediate results into a report when a step
    in working plan is completed.

    Returns:
        ToolResponse:
            The summarized draft report.
    """