A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence

📢 Updates

• 2025.07: We released a github repo to record papers related with reasoning economy. Feel free to cite or open pull requests.

📜 Table of Contents

• A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence
  • 1. Introduction
  • 2. Definitions and Foundations
  • 3. What to Evolve?
    • 3.1 Models
    • 3.2 Context
      • 3.2.1 Memory Evolution
      • 3.2.2 Prompt Optimization
    • 3.3 Tools
    • 3.4 Architecture
      • 3.4.1 Single-Agent System Optimization
      • 3.4.2 Multi-Agent System Optimization
  • 4. When to Evolve?
    • 4.1 Intra-test-Time Self-Evolution
    • 4.2 Inter-test-Time Self-evolution
  • 5. How to Evolve
    • 5.1 Reward-based Self-Evolution
    • 5.2 Imitation and Demonstration Learning
      • 5.2.1 Self-generated Demonstration Learning
      • 5.2.2 Cross-Agent Demonstration Learning
      • 5.2.3 Hybrid Demonstration Learning
    • 5.3 Population-based and Evolutionary Methods
    • 5.4 Cross-cutting Evolutionary Dimensions
    • 5.5 Other Dimensions of Self-Evolution Methods
  • 6. Where to Evolve?
    • 6.1 General Domain Evolution
    • 6.2 Specialized Domain Evolution
  • 7. Evaluation of Self-evolving Agents
  • 8. Future Directions

1. Introduction

• Large language model agent: A survey on methodology, applications and challenges

• Advances and challenges in foundation agents: From brain-inspired intelligence to evolutionary, collaborative, and safe systems

• Toward a Theory of Agents as Tool-Use Decision-Makers

• A survey on self-evolution of large language models

2. Definitions and Foundations

• Curriculum Learning for Cooperation in Multi-Agent Reinforcement Learning

• A Survey on Curriculum Learning

• Self-Evolving Curriculum for LLM Reasoning

• Continual lifelong learning with neural networks: A review

• Lifelong Learning of Large Language Model-based Agents: A Roadmap

3. What to Evolve?

3.1 Models

• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play

• Self-Challenging Language Model Agents

• Self Rewarding Self Improving

• SELF: Self-Evolution with Language Feedback

• Self-reasoning Language Models

• AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation

• Reflexion: Language Agents with Verbal Reinforcement Learning

• AdaPlanner: Adaptive Planning from Feedback with Language Models

• Self-Refine: Iterative Refinement with Self-Feedback

• Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments

• DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learning

3.2 Context

3.2.1 Memory Evolution

• Self-evolving Agents with reflective and memory-augmented abilities

• Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

• MemInsight: Autonomous Memory Augmentation for LLM Agents

• Large Language Models Are Semi-Parametric Reinforcement Learning Agents

• Agent Workflow Memory

• Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy

• Memory OS of AI Agent

3.2.2 Prompt Optimization

• Large Language Models Are Human-Level Prompt Engineers

• Large Language Models as Optimizers

• Automatic Prompt Optimization with "Gradient Descent" and Beam Search

• PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization

• REVOLVE: Optimizing AI Systems by Tracking Response Evolution in Textual Optimization

• DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

3.3 Tools

• Voyager: An Open-Ended Embodied Agent with Large Language Models

• Alita: Generalist Agent Enabling Scalable Agentic Reasoning with Minimal Predefinition and Maximal Self-Evolution

• Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLM

• From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions

• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs

• ToolGen: Unified Tool Retrieval and Calling via Generation

• AgentSquare: Automatic LLM Agent Search in Modular Design Space

• Retrieval Models Aren't Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models

• Towards Completeness-Oriented Tool Retrieval for Large Language Models

3.4 Architecture

• AgentSquare: Automatic LLM Agent Search in Modular Design Space

• Gödel Agent: A Self-Referential Framework for Agents Recursively Self-Improvement

• AlphaEvolve: A coding agent for scientific and algorithmic discovery

• TextGrad: Automatic "Differentiation" via Text

• EvoFlow: Evolving Diverse Agentic Workflows On The Fly

• Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies

• AFlow: Automating Agentic Workflow Generation

• Automated Design of Agentic Systems

• AutoFlow: Automated Workflow Generation for Large Language Model Agents

• Language Agents as Optimizable Graphs

• ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimization

• FlowReasoner: Reinforcing Query-Level Meta-Agents

• ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learning

4. When to Evolve?

4.1 Intra-test-Time Self-Evolution

• AdaPlanner: Adaptive Planning from Feedback with Language Models

• Reflexion: Language Agents with Verbal Reinforcement Learning

• Self-Refine: Iterative Refinement with Self-Feedback

• TrustAgent: Towards Safe and Trustworthy LLM-based Agents

• Self-Adapting Language Models

• Test-Time Training on Nearest Neighbors for Large Language Models

• Efficiently Learning at Test-Time: Active Fine-Tuning of LLMs

• LADDER: Self-Improving LLMs Through Recursive Problem Decomposition

• TTRL: Test-Time Reinforcement Learning

4.2 Inter-test-Time Self-evolution

• SELF: Self-Evolution with Language Feedback

• STaR: Bootstrapping Reasoning With Reasoning

• Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

• SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning

• RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learning

• DYSTIL: Dynamic Strategy Induction with Large Language Models for Reinforcement Learning

• Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation

• WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

• DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning

5. How to Evolve?

5.1 Reward-based Self-Evolution

• Vision-Zero: Scalable VLM Self-Improvement via Strategic Gamified Self-Play

• Self-Refine: Iterative Refinement with Self-Feedback

• Training Language Models to Self-Correct via Reinforcement Learning

• PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative Verifier

• Confidence Improves Self-Consistency in LLMs

• Self-ensemble: Mitigating Confidence Distortion for Large Language Models

• Self Rewarding Self Improving

• Scalable Best-of-N Selection for Large Language Models via Self-Certainty

• Can Large Reasoning Models Self-Train?

• Unsupervised Post-Training for Multi-Modal LLM Reasoning via GRPO

• Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning

• SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Development

• A Self-Improving Coding Agent

• Feedback Friction: LLMs Struggle to Fully Incorporate External Feedback

• Unified Software Engineering agent as AI Software Engineer

• OTC: Optimal Tool Calls via Reinforcement Learning

• AutoRule: Reasoning Chain-of-thought Extracted Rule-based Rewards Improve Preference Learning

• SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning

• Reward Is Enough: LLMs Are In-Context Reinforcement Learners

• Generalist Reward Models: Found Inside Large Language Models

5.2 Imitation and Demonstration Learning

• STaR: Bootstrapping Reasoning With Reasoning

• V-STaR: Training Verifiers for Self-Taught Reasoners

• AdaSTaR: Adaptive Data Sampling for Training Self-Taught Reasoners

• Enhancing Large Vision Language Models with Self-Training on Image Comprehension

• Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator

• SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning

• Bridging the Gap: Self-Optimized Fine-Tuning for LLM-based Recommender Systems

• Recursive Introspection: Teaching Language Model Agents How to Self-Improve

• Confidence Matters: Revisiting Intrinsic Self-Correction Capabilities of Large Language Models

5.3 Population-based and Evolutionary Methods

• Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agents

• Nature-Inspired Population-Based Evolution of Large Language Models

• Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models

• SPC: Evolving Self-Play Critic via Adversarial Games for LLM Reasoning

• Language Models can Self-Improve at State-Value Estimation for Better Search

• elf-Evolving Multi-Agent Collaboration Networks for Software Development

• Multi-Agent Collaboration via Evolving Orchestration

• MDTeamGPT: A Self-Evolving LLM-based Multi-Agent Framework for Multi-Disciplinary Team Medical Consultation

• Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions

6. Where to Evolve?

6.1 General Domain Evolution

• Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks

• WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning

• WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Model

• MobileSteward: Integrating Multiple App‑Oriented Agents with Self‑Evolution

• Generative Agents: Interactive Simulacra of Human Behavior

• Intelligent Virtual Assistants with LLM-based Process Automation

• UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents

6.2 Specialized Domain Evolution

• Paper Copilot: A Self-Evolving and Efficient LLM System for Personalized Academic Assistance

• Richelieu: Self-Evolving LLM-Based Agents for AI Diplomacy

• AlphaEvolve: A Learning Framework to Discover Novel Alphas in Quantitative Investment

• MDTeamGPT: A Self-Evolving LLM-based Multi-Agent Framework for Multi-Disciplinary Team Medical Consultation

• Self-Evolving Multi-Agent Simulations for Realistic Clinical Interactions

• LLMs Can Simulate Standardized Patients via Agent Coevolution

• SEW: Self-Evolving Agentic Workflows for Automated Code Generation

• AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation

• A Self-Improving Coding Agent

• QuantAgent: Seeking Holy Grail in Trading by Self-Improving Large Language Model

• Voyager: An Open-Ended Embodied Agent with Large Language Models

• Learning to Be A Doctor: Searching for Effective Medical Agent Architectures

• Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents

• Simulating Classroom Education with LLM-Empowered Agents

• One Size Doesn’t Fit All: A Personalized Conversational Tutoring Agent for Mathematics Instruction

7. Evaluation of Self-evolving Agents

• Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge

• MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Servers

• DSBENCH: HOW FAR ARE DATA SCIENCE AGENTS FROM BECOMING DATA SCIENCE EXPERTS? ICLR 2025

• ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery ICLR 2025 Code

• AppBench: Planning of Multiple APIs from Various APPs for Complex User Instruction EMNLP 2025

• MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering

• SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

• OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

• Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks

• WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

• WebArena: A Realistic Web Environment for Building Autonomous Agents

• WebWalker: Benchmarking LLMs in Web Traversal

• ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents

• xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations

• BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents

• Agent-SafetyBench: Evaluating the Safety of LLM Agents

• LifelongAgentBench: Evaluating LLM Agents as Lifelong Learners

• AgentBench: Evaluating LLMs as Agents

• GAIA: a benchmark for General AI Assistants

• TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks

• ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs code

• Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark

• API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs

• T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step ACL 2024

• ACEBench: Who Wins the Match Point in Tool Usage?

• StoryBench: A Multifaceted Benchmark for Continuous Story Visualization

• MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents ACL 2025

• Benchmarking LLMs' Swarm intelligence

• Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models

• ACPBench: Reasoning about Action, Change, and Planning

• PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change

8. Future Directions

8.1 Personalize AI Agents

• Personalization of Large Language Models: A Survey

• AutoPal: Autonomous Adaptation to Users for Personal AI Companionship

• Personalize Your LLM: Fake it then Align it

• A Survey of Personalization: From RAG to Agent

• ROUGE: A Package for Automatic Evaluation of Summaries

• Bleu: a Method for Automatic Evaluation of Machine Translation

8.2 Generalization

• Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives

• Automating Safety Enhancement for LLM-based Agents with Synthetic Risk Scenarios

• TrustAgent: Towards Safe and Trustworthy LLM-based Agents

• Foundational Challenges in Assuring Alignment and Safety of Large Language Models

8.3 Safe and Controllable Agents

• Automating Safety Enhancement for LLM-based Agents with Synthetic Risk Scenarios

• AgentDAM: Privacy Leakage Evaluation for Autonomous Web Agents

• Foundational Challenges in Assuring Alignment and Safety of Large Language Models

• Unveiling Privacy Risks in LLM Agent Memory

8.4 Ecosystems of Multi-Agents

• MDTeamGPT: A Self-Evolving LLM-based Multi-Agent Framework for Multi-Disciplinary Team Medical Consultation

• Collab-Overcooked: Benchmarking and Evaluating Large Language Models as Collaborative Agents

• Self-Evolving Multi-Agent Collaboration Networks for Software Development

• MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents

Others

• https://github.com/EvoAgentX/EvoAgentX

• https://deepmind.google/discover/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/

  • https://github.com/codelion/openevolve

🔎 Citation

To cite the research paper, you could use the following BibTeX entries.

@misc{gao2025surveyselfevolvingagentspath,
      title={A Survey of Self-Evolving Agents: On Path to Artificial Super Intelligence}, 
      author={Huan-ang Gao and Jiayi Geng and Wenyue Hua and Mengkang Hu and Xinzhe Juan and Hongzhang Liu and Shilong Liu and Jiahao Qiu and Xuan Qi and Yiran Wu and Hongru Wang and Han Xiao and Yuhang Zhou and Shaokun Zhang and Jiayi Zhang and Jinyu Xiang and Yixiong Fang and Qiwen Zhao and Dongrui Liu and Qihan Ren and Cheng Qian and Zhenghailong Wang and Minda Hu and Huazheng Wang and Qingyun Wu and Heng Ji and Mengdi Wang},
      year={2025},
      eprint={2507.21046},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2507.21046}, 
}