LLM-Driven Evolution in Core War: A Digital Red Queen Arms Race

In a fascinating twist on the classic programming game Core War, researchers at Sakana AI have developed an algorithm called Digital Red Queen (DRQ) that uses large language models (LLMs) to drive an adversarial evolutionary process. This setup simulates a continuous arms race where programs, known as "warriors," adapt and evolve to outcompete each other in a virtual machine environment.

What Changed Technically

• Adversarial Evolution with LLMs: Instead of static benchmarks, the DRQ algorithm introduces a new warrior into the multi-agent simulation in each round. This continuous influx of new opponents forces existing warriors to constantly adapt and evolve.
• Emergent Behaviors: The dynamic adversarial process leads to the emergence of increasingly robust strategies and convergent evolution, where different code implementations settle into similar high-performing behaviors.

Core War: A Turing-Complete Sandbox

Core War is a competitive programming game introduced in 1984. In this game, developers write battle programs called warriors in a specialized assembly language called Redcode. These warriors fight for control of a virtual computer by employing sophisticated strategies such as targeted self-replication, data bombing, and massive multithreading.

Key Findings

• Increasingly General Strategies: As more rounds are played, the LLM-driven evolution discovers strategies that are not only effective but also generalizable across different scenarios.
• Convergent Evolution: Different code implementations often converge to similar high-performing behaviors, mirroring biological evolution where organisms independently develop similar traits.
• Self-Modifying Code Dynamics: Because Core War is a Turing-complete environment where code and data share the same address space, the process leads to chaotic self-modifying code dynamics.

Technical Details

• DRQ Algorithm:
  • Initialization: Start with an initial set of warriors.
  • Evolution Loop:
    • Introduce one new warrior in each round.
    • Simulate battles between all existing warriors and the new warrior.
    • Select the top-performing warriors to form the next generation.
• LLM Integration:
  • Use LLMs to generate new code implementations and strategies for the warriors.
  • The LLM is trained on a dataset of successful Redcode programs to ensure it can produce viable warriors.

Example Warriors

Two notable examples of warriors produced by DRQ are:

• Ring Warrior Enhanced v9: This warrior demonstrates a robust self-replication strategy that effectively spreads its code throughout the virtual machine.
• Spiral Bomber Optimized v22: This warrior uses a sophisticated data bombing technique to disrupt and crash other programs.

Interactive Visualization

The researchers have also developed an interactive visualization tool where users can explore the Redcode of the evolved warriors. By moving the mouse cursor over different parts of the code, users can see detailed explanations and comments generated by the LLM.

Implications for AI Research

This work positions Core War as a sandbox for studying "Red Queen" dynamics in artificial systems. The continuous adversarial process provides a safe, controlled environment for analyzing how AI agents might evolve in real-world settings such as cybersecurity. By simulating these dynamics, researchers can gain insights into the emergent behaviors and strategies that arise from constant competition.

Conclusion

The Digital Red Queen (DRQ) algorithm showcases the potential of LLMs in driving adversarial evolution. The emergent behaviors and convergent evolution observed in this study offer valuable lessons for AI research, particularly in understanding how agents adapt to ever-changing threats. Core War serves as a unique platform for exploring these dynamics, providing a rich environment for both theoretical and practical insights.

For more details, you can read the technical report and access the code on the following links:

• Web Paper
• ArXiv
• GitHub