HEADLINE: Why Modular LLM Workflows Are Losing Ground to Agents

Posted Dec 17, 2025 by Benjamin Anderson

I’ve got something to get off my chest. It’s a bit embarrassing, but I need to be honest: I have a deep-seated aversion to DSPy and GEPA. This might sound harsh, especially since the people behind these tools are incredibly talented and kind. But after giving them a fair shot-I even went through the 45-minute Colab notebook and tried integrating GEPA into my open-source LLM SDK, lm-deluge-I’ve come to the conclusion that treating LLM workflows as modular programs is often a mistake. It feels backwards, rigid, and ill-suited for the most interesting tasks.

In this article, I’ll break down why I feel this way and explore whether there are any redeeming aspects of GEPA worth salvaging.

What Is DSPy?

For those new to the scene, DSPy is a toolkit created in 2023 by Omar Khattab (@lateinteraction on Twitter), known for his work on ColBERT and the concept of "late interaction" in information retrieval. The core idea of DSPy is to make AI programs more reliable by breaking them down into modular components and decoupling their function from their specific implementation.

The Modular Approach: A Closer Look

How It Works

• Modular Design: You define what you want your AI program to do using a PyTorch-like syntax with different "modules."
• Optimization: DSPy handles the optimization, testing out various few-shot examples, prompt tuning, and even fine-tuning on small datasets.

For example, if you want to summarize a long story into a single paragraph, you might start by creating a prompt like "Summarize this long story into a single paragraph" and send it to an AI model. You could then spend time refining the prompt, specifying details like sentence count and key aspects to include. However, if your budget gets slashed or a better model becomes available, all that work can become obsolete.

DSPy aims to solve this by letting you define what you want the program to do, and it handles the optimization for you. Here’s an example from DSPy’s documentation:

def search_wikipedia(query: str) -> list:
    # Define the function using modular components
    result = dspy.module('search')(query)
    return result

The Problems with Modular LLM Workflows

1. Backwards Design

• Rigidity: Once you define a module, it’s hard to adapt it to different models or tasks without starting from scratch.
• Overhead: The process of defining and optimizing modules can be time-consuming and resource-intensive.

2. Lack of Flexibility

• Model-Specific Optimizations: Different models have different strengths and weaknesses. A prompt that works well with one model might fail with another.
• Dynamic Adaptation: Agents, which can dynamically adapt to new information or changing contexts, are more flexible than static modular programs.

3. Missed Opportunities

• Emergent Behavior: LLMs often exhibit emergent behavior when given complex tasks, which can lead to innovative solutions that a rigid modular design might miss.
• Human-AI Collaboration: Agents can better facilitate human-AI collaboration by adapting to user input and context in real-time.

Is There Any Hope for GEPA?

While I’m critical of the modular approach, there are aspects of GEPA (Generalized Execution of Prompting Algorithms) that could be valuable if integrated into a more flexible framework:

• Automated Optimization: The ability to automatically test and optimize prompts is useful.
• Modular Components: Breaking down tasks into smaller, reusable components can still be beneficial, but they need to be more adaptable.

Conclusion

While DSPy and GEPA aim to make LLM workflows more reliable and modular, their rigid design often falls short in the face of dynamic and complex tasks. Agents, which can adapt and learn on the fly, are better suited for the evolving landscape of AI development. However, there’s still value in automated optimization and modular components if they’re integrated into a more flexible framework.