Research is fundamentally a search process. Its breadth and its depth have been bottlenecked by human capability, but this constraint is slowly vanishing. Frontier models are getting better at the research loop with every capability jump, and the jumps are arriving at an ever-increasing cadence. If model capability is the dominant trend, then the winning systems won’t be elaborate autonomous research systems. They’ll be the ones that let the best models explore the largest search space with the highest experimental throughput. We have found that the most effective approach is to let frontier models drive the search process, give them infrastructure they can operate on, a structured record to build on, a little human direction, and otherwise get out of the way.
State of the art on autoresearch
To that end, we picked a well-known public benchmark. Andrej Karpathy’s autoresearch task is to train a small transformer to the lowest validation loss, measured in bits per byte (lower is better), within a fixed five-minute budget on a single GPU.
Where every setup lands on val_bpb. Our confirmed 0.9016 is the median of 25 paired confirmation runs; the hollow dot is the single best draw, shown separately because one run proves little.
Our setup is straightforward: two stock coding agents (Claude Fable 5 xhigh via Claude Code, and GPT-5.5 xhigh via Codex), with access to 50 B200 GPUs, and the ability to consult GPT-5.5 Pro as an advisor. The agents ran 4,188 experiments over two days. The final recipe reaches 0.9016 val_bpb, measured as the median of 25 confirmation runs interleaved against the previous best recipe, with a best observed single run of 0.8999. Interleaved controls are exact repeats of the current best recipe, launched alongside the challenger experiments on the same pool. As far as we know, this is the best rigorously measured result on this benchmark on B200-class hardware. The published state of the art on B200 is Recursive’s 0.9109; our confirmed number sits 0.0093 below it. At the time of writing, the autoresearch@home community’s champion configuration measures 0.9055–0.9068 when reproduced like-for-like on our hardware. The range is the spread across repeated runs of that one configuration, since individual five-minute runs vary by a few thousandths.
Letting agents search
To us, the biggest revelation isn’t that the agents succeeded at this task. It’s that they did so with almost no scaffolding. We didn’t teach the models how to do research; we gave them compute, shared state, reliable execution, and let them search.
Model capability is the only part of the stack that reliably improves on its own. Systems coupled to today’s theory of how research should work have to keep adapting to that progress; infrastructure that simply lets better models search, execute experiments, maintain state, and verify results improves automatically as the models do.
That’s exactly what happened here. Over the course of this campaign, we (the humans) made perhaps a dozen meaningful decisions: choosing directions to explore, deciding what deserved another look, and occasionally correcting course. The agents made the other several thousand. We think this is an early glimpse of what research will look like when machine execution becomes effectively free. Human judgment becomes more valuable because it can steer orders of magnitude more experimentation than before. Humans stay a key piece of the puzzle; what changes is the surface area a single researcher can cover.
The whole campaign, run by run. Grey dots are discarded experiments, orange dots are kept runs, and the staircase tracks the best recipe so far, from the stock baseline at 1.0306 down to the confirmed 0.9016. Labeled steps are the promotions that moved the frontier; the long flat stretch is ≈1,300 challengers that all failed to beat the incumbent under interleaved controls.
That only works if the process stays legible. Thousands of experiments are only useful if they’re recorded well enough that a human can understand what happened and why. We believe the artifacts produced during research are at least as valuable as the final result. They are a structured record of thousands of experiments, the paths that failed, the ones that survived, and the reasoning that connected them. They capture human judgment directing machine search at a scale that has never really existed before. As model generation becomes cheaper, that record compounds. It becomes both the memory of a research campaign and the best possible training data for the next generation of research agents.
The last year has repeatedly shown the same loop across scientific and mathematical domains. As models continue improving, the bottleneck shifts away from generating ideas and toward organizing, coordinating, and remembering them.
Scaling autoresearch with hiloop
This is what we’re building at hiloop.ai: infrastructure for large-scale machine research. It runs in your cloud or hosted by us, and it isn’t specific to ML benchmarks. Autoresearch applies anywhere progress can be phrased as “propose, run, verify”: training recipes, systems tuning, simulation, any domain with a measurable objective.
If you’re pushing the frontier in your domain and want to scale up your autoresearch process, reach out.
This is only the first result from that effort. Over the coming weeks we’ll publish much more about the campaign itself: how GPT-5.5 Pro, GPT-5.5 xhigh, and Claude Fable divided work, the strategies that consistently succeeded, the thousands of experiments they proposed, the failures that shaped the search, and the infrastructure that made the loop possible.