MLExpert: Training an Agent Model for ML Engineering

Background and Context

MLExpert: Training an Agent Model for ML Engineering studies AI4AI, the idea that large language models can build and improve AI systems on their own. The authors treat this as a practical path toward recursive self-improvement. Machine learning engineering (MLE) is the most concrete test. An agent must read a task, write code, train a model, inspect the result, and adjust the plan, round after round.

Early systems relied on hand-designed workflows to orchestrate long experiment sequences. Newer systems let the model lead the interaction. The model schedules its own experiments and changes its optimization strategy based on execution feedback. This shift creates a training problem. The feedback from one experiment is the joint product of several earlier decisions, and one full MLE run contains many costly experiments.

Training must therefore pick useful supervision out of very long trajectories. It must also learn from intermediate results instead of waiting for a single score at the end of the whole run. The paper answers with a post-training recipe for autonomous, long-horizon MLE optimization, and uses it to train MLExpert, a 35B mixture-of-experts (MoE) agent model.

Deep Analysis

The recipe has three parts. First, Kaggle-MLE Factory turns Kaggle-style machine learning problems into executable tasks and supplies an interaction harness. The model decides which experiment to run next and how to revise its approach from the results. The training signal therefore comes from a real, runnable environment and not from static question answering. Second, hierarchical agentic SFT moves skills into the MLE setting. It uses outcome-oriented supervision: trajectories are selected by their final results, not imitated wholesale. This transfers the base model's general tool use and coding ability to MLE work. Third, and most important, is Windowed Interaction RL (WIRL). Between two consecutive experiment results, the model usually makes several decisions, such as reading data, changing features, tuning parameters, and writing scripts. WIRL treats that whole stretch as one execution window. Every turn inside the window shares the same outcome-based reward.

What is WIRL, and why does it help long-horizon MLE training? WIRL, or Windowed Interaction RL, assigns credit at the level of one experiment. It groups the multi-turn decisions between two experiment results into a window with a shared outcome-based reward. This is easier to define than a per-turn reward and denser than one reward for a whole trajectory. Experiments are expensive, so the authors add asynchronous collection and policy correction. The policy can be updated before a trajectory ends, using windows that have already finished. The policy correction step handles the data staleness that asynchronous collection introduces.

Industry Impact

The headline result is on MLE-Bench Lite. MLExpert lifts the base model's average medal rate from 37.9% to 72.7%. That is nearly double. It puts the 35B MoE model on par with larger frontier models such as Kimi-K3 and GLM-5.3. The gain appears to come from post-training aimed at long-horizon experimentation, not from added parameters. The paper also tests transfer. MLExpert improves on NatureBench Lite, on agent scaffold optimization, and on algorithm optimization. These tasks differ from Kaggle-style modeling contests, yet each needs the same loop: propose a solution, run it, read the feedback, and improve. The results suggest the model learned a transferable optimization skill and not a memorized benchmark.

The abstract gives no ablation numbers. It does not say how much hierarchical SFT, WIRL, or asynchronous correction each contribute. Readers who need those splits must check the experiment sections of the full paper. The design carries lessons beyond MLE. Splitting a trajectory at feedback nodes is a general idea. Other slow-feedback domains, such as scientific experiment automation, system tuning, and automated research, could reuse it. For practitioners, a 35B model that approaches much larger models on MLE tasks points to lower inference cost for internal systems that handle automatic modeling, tuning, and scaffold optimization.

Outlook

MLExpert is one concrete piece of the AI4AI picture: a model that gets better at improving machine learning systems. The results sit on benchmark tasks, and the abstract supports claims only about those benchmarks. Medal rate on MLE-Bench Lite is a proxy for real engineering work, and the transfer gains on NatureBench Lite, scaffold optimization, and algorithm optimization are reported as improvements, not as parity with frontier models.

Several questions remain for follow-up work. The first is how much each component contributes, which only ablations can show. The second is whether windowed credit assignment holds up when experiments are cheaper or when one decision affects results many windows later. The third is how well the method scales with longer runs and larger base models.

Stronger autonomous improvement also raises the bar for evaluation reliability, experiment cost control, and safety boundaries. The paper leaves these issues to later research. For now, WIRL and the Kaggle-MLE Factory pipeline offer a clear, testable template for training agents on expensive, long-horizon tasks.