ERNIE Evolve: Toward Automated Research for Real-World AI R&D

AI R&D depends on repeated experimentation and analysis. Researchers have to choose which ideas to pursue, design and run experiments, distinguish real improvements from noise, and use what they learn to guide the next experiment. AI systems are increasingly capable of carrying out this work, with automated research already showing promise in machine learning, mathematics, and chip design. Applying these systems to improve the AI that powers them creates a recursive self-improvement (RSI) loop. For AI R&D teams, this is becoming an important source of competitive advantage, with the potential to accelerate the development of better models.

ERNIE Evolve is our first step toward this goal. It is an automated research system that works toward explicit objectives under defined constraints. Its scope covers both choosing what to investigate and developing better implementations through experiments. It pursues promising ideas over multiple iterations and adjusts its research direction as evidence accumulates. Given a code repository, an optimization objective, and evaluation criteria, ERNIE Evolve modifies code and runs experiments autonomously in isolated environments. Independent evaluations determine which improvements to retain, while findings from each experiment inspire later research.

We first applied ERNIE Evolve on three public AI R&D tasks: NanoChat AutoResearch, NanoGPT Speedrun, and SOL-ExecBench, covering training algorithms, training speed, and GPU kernel optimization. ERNIE Evolve improved on the baselines in all three tasks and outperformed the strongest public results we compared against on NanoChat AutoResearch and NanoGPT Speedrun. We then applied it to inference optimization for the in-development ERNIE-5.5-Flash. The resulting changes passed production-grade validation and delivered substantial performance gains, showing that automated research can produce practical improvements in real AI development.

Building an automated research system like ERNIE Evolve requires giving AI a working environment aligned with that of human researchers, which calls for substantial changes to existing infrastructure. Automated research systems are built on AI-native infrastructure, and they are also becoming a new form of infrastructure themselves. As RSI scales across real-world R&D, it could drive improvements in AI capabilities at an unprecedented rate.

ERNIE Evolve

ERNIE Evolve organizes research through two nested loops: the outer loop decides what to investigate, while the inner loop develops and tests improvements within each research direction. Drawing on past experience and existing knowledge, the outer loop proposes directions and assigns each to a research lane. Multiple lanes run in parallel, each maintaining its own tree of candidate code versions. Within a lane, the inner loop repeatedly builds on an existing version, modifies the code, and submits the new candidate for independent evaluation and review. Code changes and experimental results are recorded throughout the process to inspire later research.

• Director selects the next research direction and a starting version. It draws on the Experience Bank, which stores candidates, code changes, and results from previous experiments, and the Knowledge Base, which contains papers, open-source code, and domain knowledge. The starting point can be the original code, the best version found so far, an intermediate version, or the latest commit on the upstream main branch.

• Selector guides each iteration within a research lane. It chooses a candidate from the lane’s tree and specifies the hypothesis to test next. Its decisions take into account not only scores, but also code changes, reasons for failure, and the outcomes of previous decisions.

• Executor implements the Selector’s plan in an isolated environment, modifying the code to produce a new candidate version.

The Evaluator and Reviewer determine whether a candidate is accepted. The Evaluator is a fixed evaluation program that measures performance and checks correctness on real workloads. It is kept outside the repository being optimized and is not visible to any agent. Candidates with changes outside the permitted scope are excluded from evaluation, and performance gains are accepted only when they exceed the noise threshold.

The Reviewer examines the code changes alongside the evaluation results. It checks whether the candidate follows the intended research direction and whether the reported gains come from exploiting weaknesses in the evaluation. A candidate that departs from the intended direction cannot be selected as the best version and one that hacks an evaluation loophole is discarded.

The Director determines which line of research to pursue, while the Selector determines which version to evolve within that line. ERNIE Evolve can keep improving a promising approach and use experimental results to decide when to change direction. Only improvements passed independent evaluation and review can be retained. Code changes are inherited through the candidate tree, while recorded findings guide future decisions, allowing individual experiments to shape the broader research process.

Testing ERNIE Evolve on Public AI Benchmarks

We applied ERNIE Evolve on three public AI R&D tasks: NanoChat AutoResearch, NanoGPT Speedrun, and SOL-ExecBench. These cover training algorithms, training speed, and GPU kernel optimization. All three have public evaluation protocols and published results from community researchers and other automated research systems, providing established baselines for comparison. Across the three metrics reported for NanoChat AutoResearch and NanoGPT Speedrun, ERNIE Evolve surpassed the best publicly verified results available at the time of evaluation. The optimized results of ERNIE Evolve on these tasks have been open-sourced on Github.

Benchmark	NanoChat AutoResearch	NanoGPT Speedrun Track 1	NanoGPT Speedrun Track 3	SOL-ExecBench
Task	Training algorithm	Training acceleration	Training acceleration	GPU kernel optimization
Metric	Validation BPB ↓	Time to target loss ↓	Steps to target loss ↓	Mean SOL ↑
Best public result	0.8878	39.9 s	2,690 steps	0.827
Recursive	0.9109	77.5 s	—	0.676
Hyra-1.0	0.9015	76.4 s	—	0.736
ERNIE Evolve	0.8763	38.9 s	2,590 steps	0.819

Table 1: Comparison of results on public tasks. Bold indicates the best result; “—” means no result was reported. For the best results, NanoChat AutoResearch uses the result reported in Rekursiv's Auto²-Research report, NanoGPT Speedrun and SOL-ExecBench use the best results on their official leaderboards, with SOL-ExecBench evaluated using Evaluation Stack v1.1. The comparison is against Hyra-1.0, as Hyra-2.0 has not reported results on these tasks.

On NanoChat AutoResearch, ERNIE Evolve had to balance model architecture, optimizer design, learning-rate schedules, and execution efficiency within a five-minute training budget on a single GPU. With the baseline already heavily optimized by the community, architecture changes alone offered limited room for improvement. ERNIE Evolve added two prediction branches alongside the neural network. One uses n-gram statistics from the full training corpus; the other retrieves training documents that match the current context and uses their continuations to make predictions. A gate adjusts the mixture weights based on each branch’s performance at earlier token positions. This allows information from the training corpus to contribute to predictions even when the network has not had enough time to learn it.

To keep the additional overhead low, the system used a sparse optimizer that updates the large input table without storing momentum state. It also fused softcap and cross-entropy into a single Triton kernel and overlapped CPU table construction with GPU training. The final version achieved a validation bits-per-byte (BPB) score of 0.8763.

NanoGPT Speedrun has undergone more than two years of community optimization, with over 90 successive improvements, leaving little room for further gains. On Track 1, ERNIE Evolve focused on reducing the execution time of each training step. Its changes improved communication overlap, backpropagation, and output-layer gradient computation. They addressed serial waits, hidden data movement, and expensive gradient calculations that remained in an already mature training setup. The final version reached the target loss in 38.9 seconds, and repeated runs confirmed that it met the loss requirement. On Track 3, ERNIE Evolve improved the optimizer’s update rule to reduce the number of steps needed to reach the target loss. The final approach combined findings from two search branches: decomposing matrix weights into direction and magnitude, and applying EMA-Nesterov extrapolation based on an exponential moving average of past parameter displacements. In the baseline optimizer, normalization constrains persistent update directions and oscillating ones to similar step-size scales. The extrapolation allows persistent movement to accumulate across steps, helping the optimizer advance further along those directions.

Combining these mechanisms exposed an implementation conflict. The optimizer reconstructs weights from its internal state, so extrapolation applied directly to the weights would be overwritten. ERNIE Evolve resolved this by applying the extrapolation to the direction and magnitude parameters instead. It then added a learning-rate floor and weight averaging near the end of training to improve late-stage convergence. Together, these changes reduced the number of steps to the target loss to 2,590.

SOL-ExecBench contains 235 kernel optimization problems drawn from real models, each requiring an implementation that is both correct and fast. In the more than seven months since the official leaderboard launched, over 700 developers have made nearly 50,000 submissions. Records continue to improve, making further gains increasingly difficult. ERNIE Evolve achieved an average SOL score of 0.819, ahead of Recursive and Hyra-1.0 and close to the leading score of 0.827.

These tasks span different layers of AI development and provide different kinds of feedback. ERNIE Evolve used the same research loop across all three, without adapting the system to a particular metric or code structure. It found further improvements in heavily optimized code, adjusted its search based on experimental results, and used both successful and failed experiments to guide later work. Its contribution extends beyond code generation: it connects the choice of research direction, experiment execution, evaluation, and accumulated findings into a process that can continue across many iterations.

From Benchmarks to Real-World AI R&D

Following our work on public research tasks, we applied ERNIE Evolve to the ERNIE team’s ongoing inference optimization for ERNIE-5.5-Flash. This was our first step toward integrating automated research into the team’s day-to-day engineering work. We focused on three tasks: low-latency inference on Kunlunxin P800, deterministic inference throughput on GPUs, and pushing the limit of decoding speed. Each addressed an active engineering need. ERNIE Evolve’s changes had to pass the same load tests and correctness checks as changes submitted by engineers.

These tasks introduced several challenges beyond those of the public tasks:

• Stronger baselines. Most baselines had already been optimized by engineers for the target workload. Further gains depended on finding remaining inefficiencies in kernels, communication paths, and scheduling.

• Tighter constraints. Changes had to be suitable for production code and services. They were limited to specified modules and had to preserve the required determinism while meeting existing accuracy and latency requirements.

• Noisier feedback. A change’s effect often became clear only in an end-to-end load test. Individual gains could be comparable to measurement noise, making it difficult to distinguish a useful change from a fluctuation.

• More complex environments. Moving from a single training script or kernel to a full inference system meant working across inference engines, kernel libraries, hardware plugins, and service startup scripts. To evaluate a change in this environment, ERNIE Evolve had to launch the service across multiple accelerators with these components correctly configured, then run performance and correctness checks. Errors in this setup could cause experiments to fail or make their results unreliable.

Low-Latency Inference on Kunlunxin P800

ERNIE-5.5-Flash could already run on Kunlunxin P800, but was not yet making full use of the available optimized execution paths. The task was to improve inference speed for low-latency workloads while keeping the model and deployment setup unchanged. Changes were restricted to the Kunlunxin plugin and had to pass all correctness checks.

For this workload, each decoding step involved little computation. Much of the time went to Python calls, temporary tensor allocations, and host-device synchronization, repeated at every layer and every step. Across four rounds of research, ERNIE Evolve raised generation speed to 623.6% of the baseline while passing all correctness checks.

The first round enabled optimized paths that were already available. ERNIE Evolve replaced the many small operators used for manifold-constrained hyper-connections (mHC) with Kunlunxin’s native fused implementation and enabled speculative decoding through the model’s built-in multi-token prediction layer. Together, these changes increased generation speed to 282.9% of the baseline.

The second round focused on host-side scheduling and synchronization. Small matrix multiplications bypassed wrappers that were unnecessary for inference and called the underlying implementation directly. The system also removed the wait for device-to-host synchronization during verification, reused indices and intermediate data across layers, and switched attention to a native implementation. The cumulative gain reached 588.8% of the baseline. One alternative failed: replacing specialized communication with general-purpose collectives reduced speed by 14.0%, so it was dropped.

The third round cached indices and work buffers for mixture-of-experts (MoE) computation, but improved speed by only about 1.4% over the previous round. This prompted ERNIE Evolve to measure the time spent in each stage again. In the fourth round, runtime checks revealed that automatic communication selection was still sending decoding requests through the standard path. Fixing the dispatch logic and sizing the communication buffers for the actual workload enabled DeepEP’s low-latency path, bringing the total gain to 623.6% of the baseline.

Increasing Throughput for Deterministic Inference

For our RL training workload, inference had to satisfy strict consistency requirements. A request’s generated tokens and log probabilities had to be bitwise identical regardless of which other requests shared its batch. The log probabilities also had to match those recomputed over the full sequence by the training system. SGLang’s deterministic mode fixes reduction order to support these requirements, but this limits GPU utilization and slows inference.

The task was to increase ERNIE-5.5-Flash throughput while preserving these guarantees. Changes were restricted to settings and implementations related to deterministic execution, with no regression in accuracy or latency. Common speedups that changed reduction order, including batch-dependent kernel selection, atomic additions, and adaptive communication, were ruled out.

The engineering team had already optimized this baseline for deterministic inference. Many remaining changes offered only 1%–2% individually, comparable to measurement noise. After roughly three days of research, ERNIE Evolve increased throughput by 33.7%, reduced average time per output token by 26%, and reduced time to first token by 12%, with determinism preserved.

The search began with profiling. Deterministic sampling accounted for about 14% of GPU time per step, making it the largest bottleneck. ERNIE Evolve replaced it with a Triton kernel that processes vocabulary chunks in parallel. This delivered the largest individual gain. Improvements to sparse attention and small-batch matrix multiplication then brought the cumulative gain to 20.6%.

The next direction examined settings for deterministic execution. Changing a MoE communication flag alone produced no gain distinguishable from noise, but pairing it with the corresponding kernel changes improved performance. Adjusting the hand-tuned mHC partition setting from 8 to 32 yielded another gain, bringing the total to 32.6%.

The final direction tested whether limited parallelism was a broader problem. Many kernels used fixed partition counts in deterministic mode, while each GPU handled very little data per decoding step. ERNIE Evolve investigated whether these kernels were leaving too many compute units idle. Tests did not support this hypothesis for most of the kernels examined. Only a few changes were retained, including parallelizing mHC post-processing across two dimensions, bringing the final throughput gain to 33.7%.

Pushing the Limits of Decoding Speed

This task started from an ERNIE-5.5-Flash service already tuned for fast single-request decoding on GPUs. The service used speculative decoding: a draft model proposed several tokens, and the main model verified them in one pass. ERNIE Evolve focused on reducing the time taken by each step in this setup.

The engineering team had already optimized cross-GPU communication and reduced the gaps between kernels, leaving no single dominant bottleneck. The remaining time was spread across many small kernels in main-model verification and draft-model generation, each processing only a few tokens. Some useful changes improved speed by just 0.3%–0.5% individually, less than the measurement noise. ERNIE Evolve ultimately increased decoding speed by a further 14.9%, with accuracy unchanged from the baseline.

ERNIE Evolve explored seven research directions, three of which produced improvements. The first gains came from MoE and mHC. When the first matrix multiplication in the MoE block ran alongside routing computation, the two operations were assigned compute units inefficiently. Rebalancing that allocation improved performance. The system then modified mHC post-processing to produce the FP8-quantized input needed by the next layer, removing a separate quantization kernel. Together, these changes improved speed by 5.4%.

Three other directions in the same round investigated long-context attention, cross-GPU communication, and kernel gaps or host overhead. Measurements ruled out these suspected bottlenecks. A cross-GPU communication operation was already close to the latency floor of an NVLink round trip, while gaps between kernels accounted for only 2%–4% of each step.

The next gains came from main-model matrix multiplication and MoE scheduling. For inputs with fewer than eight rows, zero padding accounted for half of the operand data in the original FP8 matrix multiplication. ERNIE Evolve reformulated the computation by swapping the operand roles of the weight and input matrices, then used native FP16 matrix-multiplication instructions. It also changed the MoE expert scheduler to use a two-level index instead of scanning all experts sequentially for each group. The scheduler change delivered the largest individual gain. Along with several kernel fusions, these changes brought the cumulative improvement to 13.2%.

The final gains came from the draft model, which remained on the critical path. ERNIE Evolve fused per-position sampling and cross-GPU reduction into a single kernel and combined computations in draft attention. These changes brought the total decoding-speed improvement to 14.9%.

The Path Forward

Across public benchmarks and real-world inference optimization, ERNIE Evolve has demonstrated that it can sustain research and deliver effective improvements on R&D tasks with clear objectives and verifiable results. These gains come from a continuous research process that links research directions, code versions, experimental results, and subsequent decisions across iterations.

Making RSI run more smoothly and reliably requires continued development of infrastructure that enables AI to participate fully in research. Human researchers can explore a codebase, choose research questions and starting versions, design and run experiments, revise hypotheses based on results, retain useful changes, abandon unsuccessful directions, and carry what they learn into the next iteration. AI needs the same scope for action and a working environment that provides access to the code, tools, evaluations, and feedback needed to carry out this process.

Research history is an equally important part of that environment. Each direction should remain linked to the code versions, results, and failure reasons it produced. Those records need to be available when the system plans its next experiment, so it can build on previous work and avoid repeating unsuccessful approaches without a new reason to try them.

Optimizing inference infrastructure is our first application of RSI to real-world AI R&D. This work also shows how the same research loop can extend to other stages of AI development. As the infrastructure matures, we plan to apply this loop to training system optimization, then use model capability metrics to guide automated hypothesis generation, experiment design, and validation. Beyond improving models, we aim to enable AI to improve benchmarks and evaluation suites themselves by identifying capability gaps and generating and refining evaluation tasks, so that evaluation evolves alongside the research.

Through these efforts, we aim to make automated research a reliable part of everyday AI development. At the same time, RSI must expand under human control and remain interruptible, operating within clear boundaries for goals, permissions, and resources with continuous safety evaluation and human oversight. Its scope should grow only while people can still understand, supervise, and stop the system.