DiscoveryRSI: Teaching Models to Propose Hypotheses

Background and Context

A key step in scientific discovery is turning a valuable hypothesis into research that someone can act on. Writing a novel-sounding guess is the easy part. Pushing that guess through revision, investigation and review until it can be tested is the hard part. Learning from this process is difficult because it keeps evolving. Many research agents are judged only on their final output. They learn little from the way a hypothesis grows and changes over many rounds.

The paper behind DiscoveryRSI starts by defining what a valuable, testable hypothesis is. The authors split it into parts that must all be present. It is an evidence-based claim about an important uncertainty. It comes with a feasible investigation plan. It states clear decision criteria, meaning which result would support the claim and which would refute it. It also lists alternative explanations, so the researcher does not fixate on a favored answer.

On top of this definition, the authors propose the DiscoveryRSI framework. It lets a model improve its hypothesis-proposing ability through recursive self-improvement. They also introduce DFM-Bench to measure that improvement. The work targets four life-science domains.

Deep Analysis

DiscoveryRSI first uses multi-agent evolutionary search to build a hypothesis tree. Several agents propose, criticize and revise hypotheses around the same research direction. Each branch of the tree stands for a different development path. The chosen investigations are then compiled into executable Harbor tasks. Research therefore stops being a text description. It becomes something that can run and leave a record. Three mechanisms drive the recursive learning. Hypothesis credit optimization attributes scientific evaluation results to specific research contributions recorded in the documents. The model can see which revision actually raised hypothesis quality, instead of spreading credit across a whole trajectory. Harness replay revisits existing research runs, finds effective strategies and recurring failure modes, and uses them to improve the research workflow itself. Rediscovery applies accepted improvements to new investigations, and those investigations produce fresh experience for the next round. Together they form a loop: search produces experience, experience improves the workflow and the model, and the improved system searches again.

How does DiscoveryRSI measure whether a hypothesis is good? It uses DFM-Bench, which scores quality at four levels: hypotheses, revisions, development paths and research runs. This checks more than one final score. It also asks whether research accumulates progress across several revision rounds. In automatic evaluation across the four life-science domains, hypothesis quality rose from 4.86 for the base model to 5.39 after two rounds of self-improvement. The largest gains came in falsifiability and data resolvability. That fits the framework design, which stresses decision criteria and executable investigations. Expert review adds a second signal. In preliminary human evaluation, DiscoveryRSI-27B scored higher than GPT-6 Luna on hypothesis importance and on whether claims are sound and bounded by evidence. Human scoring also showed larger improvements along the research path. The comparison with later literature is the most telling. Using research published after the models' cutoff dates as the reference, the hypotheses of DiscoveryRSI-27B matched published work at a rate of 6.3%. GPT-6 reached 2.5% and Claude Opus 5 reached 2.3%. The human evaluation is still preliminary, and its sample size and coverage need more validation.

Industry Impact

The first effect is practical. The paper turns a good hypothesis from a matter of taste into something that can be defined and measured. Metrics such as falsifiability and data resolvability give research agents a clear optimization target. Teams building automated research tools can reuse the four-part definition as a checklist for their own outputs.

The second effect concerns model size. A 27B model that learns recursively from its own research process reached hypothesis quality that competes with, and on some expert measures beats, a larger model such as GPT-6 Luna. For open-source groups with limited compute, this is a useful signal. Better process learning may matter as much as raw scale in this task.

The third effect is on engineering. Compiling investigations into executable Harbor tasks, and attributing credit at the level of individual research contributions, offers a design pattern for industrial research pipelines. Examples include drug target screening or experiment design, although the paper evaluates only life-science domains and does not test such deployments. DFM-Bench also gives researchers a multi-level ruler. They can compare how much progress different agents accumulate, not only the quality of a single output.

Outlook

The 6.3% match rate shows how far the field still is from reliably foreseeing real discoveries. The rate is well above the 2.5% of GPT-6 and the 2.3% of Claude Opus 5, yet most hypotheses still do not correspond to later published work. Readers should treat the figure as a relative gain, not as proof of dependable prediction.

Two open questions follow from the paper. First, the evaluation covers four life-science domains. Whether the same loop of tree search, Harbor compilation and recursive learning works in other disciplines remains unanswered. Second, the human evaluation is preliminary. Larger expert panels and wider coverage would show whether the gains in importance and evidence boundaries hold up.

The next test is whether further self-improvement rounds keep raising hypothesis quality beyond the two rounds reported, from 4.86 to 5.39. If they do, DiscoveryRSI would show that process-level credit assignment can sustain growth. If they do not, the limits of the loop will themselves be a useful finding for the field.