Building Effective AI Agent Systems with a Two-Tier Model

I’ve been diving deep into integrating AI agents into UserJot, our feedback, roadmap, and changelog platform. The goal was to analyze customer feedback at scale, identify patterns across hundreds of posts, and auto-generate changelog entries. After weeks of experimenting, reverse engineering tools like Gemini CLI and OpenCode, and running numerous tests, I’ve landed on a robust two-tier model that works well in practice.

The Two-Tier Agent Model That Actually Works

Forget complex hierarchies; you need exactly two levels:

• Primary Agents: Handle the conversation. They understand context, break down tasks, and communicate with users. Think of them as project managers who never write code.
• Subagents: Do one thing well. They receive a task, complete it, and return results. No memory, no context, just pure function execution.

I tried three-tier and four-tier systems where agents talked to other agents, but they all broke down. The two-tier model is simpler and more reliable.

Here’s the architecture I settled on:

User → Primary Agent (maintains context)
         ├─→ Research Agent (finds relevant feedback)
         ├─→ Analysis Agent (processes sentiment)
         └─→ Summary Agent (creates reports)

Each subagent operates in complete isolation, while the primary agent handles all the orchestration.

Stateless Subagents: The Most Important Rule

Every subagent call should be like a pure function with the same input producing the same output. No shared memory, no conversation history, no state. This might seem limiting at first, but it offers several key benefits:

• Parallel execution: Run multiple subagents simultaneously without them interfering with each other.
• Predictable behavior: The same prompt always produces similar results.
• Easy testing: Test each agent in isolation.
• Simple caching: Cache results by hashing the prompt.

Here’s how I structure subagent communication:

// Primary → Subagent
{
  "task": "Analyze sentiment in these 50 feedback items",
  "context": "Focus on feature requests about mobile app",
  "data": [...],
  "constraints": {
    "max_processing_time": 5000,
    "output_format": "structured_json"
  }
}

// Subagent → Primary
{
  "status": "complete",
  "result": {
    "positive": 32,
    "negative": 8,
    "neutral": 10,
    "top_themes": ["navigation", "performance", "offline_mode"]
  },
  "confidence": 0.92,
  "processing_time": 3200
}

No conversation history or state, just task in and result out.

Task Decomposition: How to Break Things Down

You have two effective strategies for decomposing tasks:

• Vertical decomposition for sequential tasks:

  "Analyze competitor pricing" →
    1. Gather pricing pages
    2. Extract pricing tiers
    3. Calculate average prices
  

• Horizontal decomposition for parallel tasks:

  "Generate a comprehensive report on user feedback" →
    1. Research Agent: Find relevant feedback posts
    2. Analysis Agent: Process sentiment and categorize feedback
    3. Summary Agent: Create a structured report
  

Implementation Notes

• Primary Agents: Use a stateful orchestration framework like Apache Airflow or Argo Workflows to manage the flow of tasks.
• Subagents: Implement subagents as microservices or serverless functions for scalability and isolation.
• Caching: Use a distributed cache like Redis to store and retrieve results based on hashed prompts.

By following these principles, you can build robust AI agent systems that handle complex tasks efficiently and reliably. I’ve pushed a basic version of this system to production as a beta, and it’s mostly working well so far.