AutoDesign: A New Paradigm of Meta-Framework Optimization for Long-Horizon Agent Design

Background and Context

The transition of artificial intelligence from single-task execution to complex, long-horizon agent processes has introduced significant challenges in computer vision and natural language processing. A core difficulty lies in efficiently converting multimodal sources into condensed, structured media outputs. Traditional paradigms often treat this conversion as a static mapping task, lacking the capacity for dynamic alignment with design priors or the accumulation of reusable experience through empirical exploration. This static approach results in a missing self-improvement mechanism, causing systems to struggle in complex scenarios where iterative refinement is required.

To address these limitations, the AutoDesign framework introduces a meta-harness optimizer, aiming to construct a model-framework system capable of recursive self-improvement. The framework is designed to align with human design intuition while guiding code agents to iteratively optimize the execution environment based on rollout feedback. This approach facilitates the continuous accumulation of experience and a spiral rise in capability during long-horizon tasks. By moving beyond static paradigms, AutoDesign provides a dynamic optimization perspective that fundamentally alters how agent design is approached in complex media generation tasks.

Deep Analysis

The technical architecture of AutoDesign revolves around two core mechanisms: meta-framework optimization and recursive improvement by code agents. The system begins by constructing an initial harness structure. The meta-framework optimizer then intervenes by analyzing the rollout feedback generated by the code agent during task execution. This analysis identifies bottlenecks and deviations within the current harness. Based on these insights, the optimizer guides the code agent to recursively modify and restructure the harness, gradually bringing it closer to the ideal state required by human design priors.

This process is not a one-time adjustment but a continuous closed-loop iteration. The agent executes the task, generates feedback, the optimizer analyzes that feedback, and the agent subsequently modifies the harness before executing the task again. Through this recursive mechanism, the system can autonomously discover and correct logical flaws and aesthetic deviations in the design without relying on extensive human annotation. This technical path emphasizes the critical role of experience accumulation in agent evolution, allowing the system to learn from every interaction and form reusable design knowledge rather than relying solely on the static parameters of pre-trained models.

Industry Impact

To rigorously evaluate the effectiveness of AutoDesign, the research team developed the PosterBench benchmark. This benchmark consists of a Main Track covering five disciplinary fields with 100 papers, and a PosterBench-mini subset of 10 papers for controlled evaluation. Experimental results on the PosterBench Main Track show that AutoDesign achieved a top score of 78.32. This score surpasses the closed-source commercial system Claude Design by 7.45 points, demonstrating superior performance in complex academic poster generation tasks. The significant margin indicates that the framework effectively handles the intricacies of converting dense academic content into visually structured media.

Further ablation experiments across seven different controlled code agent model configurations reveal that the integrated DesignHarness significantly boosts performance. The average PosterBench score increased from 54.99 to 67.39, representing a relative improvement of 12.4%. These results prove that the gains from meta-framework optimization are not dependent on a specific underlying model but stem from the optimization of the harness itself. This model-agnostic nature suggests that the framework can be applied broadly across various agent architectures without requiring fundamental changes to the base models.

Outlook

In fully autonomous long-horizon loop tests, the system demonstrated remarkable efficiency and cost-effectiveness. It completed 253 tool calls and 11 rounds of editing in just 40 minutes, with a total cost of under $3. The final generated posters reached the quality level of average conference posters in human evaluations. Furthermore, in blind system tests, the outputs received the highest human preference scores. These metrics validate the dual advantages of AutoDesign in both operational efficiency and output quality, highlighting its potential for scalable deployment in resource-constrained environments.

The introduction of AutoDesign holds profound implications for the open-source community and industrial implementation. It provides a generalizable paradigm for long-horizon agent optimization that extends beyond poster generation to other multimodal content creation tasks, such as slide production and video script generation. The release of the PosterBench benchmark fills a gap in academic evaluation for structured media generation, offering a standardized platform for fair comparison and rapid iteration. In industrial applications, the framework's low cost and high efficiency make it highly commercially viable for sectors requiring rapid generation of high-quality design content, such as marketing and educational publishing. The recursive self-improvement mechanism offers a new pathway for building more autonomous and adaptive AI systems, potentially reshaping human-machine collaborative design workflows.