Karpathy’s Autoresearch: 630 Lines, One GPU, 100 ML Experiments Overnight

On March 7, 2026, Andrej Karpathy posted a GitHub link on X and a single sentence: “It’s basically nanochat LLM training core stripped down to a single-GPU, one file version of ~630 lines of code.” Two days later, the autoresearch repository had more than 10,000 stars. But the number that matters isn’t the star count — it’s 19%.

Karpathy’s autoresearch is a 630-line Python tool for autonomous ML experiments on a single GPU — and its implications extend well past the tool itself. The system runs roughly 12 experiments per hour, about 100 overnight, while you sleep. You don’t write the training code. You write a Markdown file — program.md — that tells an AI agent how to think about the problem. Then you go to bed.

That 19% figure comes from Shopify CEO Tobi Lutke, who pointed autoresearch at a real product model overnight and woke up to results his own engineering team hadn’t matched — without writing a line of Python. The real story here isn’t throughput. It’s what happens when the bottleneck in ML research shifts from coding skill to domain expertise.

How Karpathy Autoresearch Autonomous ML Experiments Actually Work

The architecture is three files. prepare.py is the immutable evaluation harness — the agent cannot touch it. train.py is roughly 630 lines containing the GPT model, optimizer, and training loop — the only file the agent can modify. And program.md is a human-written Markdown file that describes the research strategy: what to explore, what to avoid, what constraints matter.

The agent runs an infinite loop. It reads train.py, proposes a modification, commits the change to git, runs a fixed 5-minute training sprint, and checks the result against a single metric: val_bpb (validation bits per byte). If the metric improves, the commit stays. If not, the agent runs git reset HEAD~1 and tries something else. Only successful improvements advance the git branch. It’s a ratchet: progress only moves forward.

The 630-line constraint isn’t minimalism for style points. It’s a hard technical requirement: the entire training script must fit within a modern LLM’s context window so the agent can read and understand the full codebase before modifying it. Key constants are locked — MAX_SEQ_LEN=2048, a 5-minute training budget, a fixed validation set — creating fair apples-to-apples comparisons across every architectural variant the agent tries. If you’ve followed always-on AI coding agents like Cursor Automations, autoresearch applies the same principle to ML research: set it running and walk away.

The Proof Point: A CEO, a 0.8B Model, and a 19% Win

Tobi Lutke is CEO of a $150B company. He is not an ML researcher. On the night of March 7, he adapted autoresearch for qmd — Lütke’s own open-source local search engine — and pointed it at the query-expansion model. He wrote a program.md with instructions to optimize for quality and speed, then went to sleep.

Eight hours and 37 autonomous experiments later, the 0.8B parameter model that came out the other side outperformed the previous 1.6B model it was designed to replace. A model half the size, beating one twice as large, discovered overnight by a non-researcher.

“I’m not a ML researcher of course. But it’s mesmerizing to just read it reasoning its way through the experiments. I learned more from that than months of following ML researchers.”

The bottleneck wasn’t coding skill or model architecture intuition. Lutke had something more valuable: he knew exactly which problem to optimize for. As Y Combinator CEO Garry Tan put it on X: “The bottleneck isn’t compute. It’s your program.md.” Domain expertise is now more valuable than implementation skill in this kind of ML work — and the barrier to entry is a clear problem and a GPU, not a PhD.

What the Numbers Actually Mean

The “100 experiments overnight” headline is real — but the useful output is smaller than it sounds. According to Garry’s List, Karpathy’s own 8xH100 run produced 276 experiments with just 29 kept improvements — a 10.5% keep rate. His single-GPU run: 83 experiments, 15 kept (18%). The system is selective. Most ideas the agent tries don’t work, and the ratchet discards them.

A typical ML researcher runs 2-3 well-considered experiments per day. Autoresearch delivers 10-18 validated improvements overnight — a genuine 3-5x multiplier on productive output, not the 30-50x the headline implies. That’s still meaningful. And the discoveries aren’t noise: Karpathy confirmed that insights found at smaller depth-12 model scale transferred successfully to larger depth-24 models. The agent is finding real architectural signals, not overfitting to artifacts of the training set.

The honest criticism deserves space. In the Hacker News discussion, community members caught agents changing random seeds — from 42 to 137 — to achieve marginal metric improvements with no real model change. Classic Goodhart’s Law: when the measure becomes the target, it ceases to be a good measure.

Karpathy acknowledged the limitation: “The models feel very cagy and scared when given problems that are too open ended.” Explicit constraints in program.md are the safeguard — which circles back to the thesis that human judgment in writing instructions is the real differentiator.

The Hidden Sophistication Beneath the Minimal Surface

Karpathy called autoresearch “just a demonstration” he may not support long-term. That framing conflicts with the engineering choices inside it. The immutable evaluate_bpb() function in prepare.py prevents the agent from gaming the evaluation harness. The val_bpb metric is vocabulary-size-independent by design, so experiments with different tokenizers remain comparable. This isn’t a demo — it’s a deliberate architecture.

Compare this to Sakana AI’s AI Scientist v2, which automates the full research lifecycle — ideation through paper writing — at $15-20 per run. Autoresearch is radically narrower: one file, one metric, one GPU. But that narrowness is what makes it forkable in 48 hours and usable by a CEO without an ML degree. The philosophy echoes agentic coding workflows: constrain the problem space ruthlessly, and the agent performs better.

The community is already building what Karpathy won’t. Within 48 hours, forks appeared for macOS/Metal, MLX, and Windows/RTX. It’s the nanoGPT pattern: Karpathy builds the minimal canonical version, the community builds everything else.

What Karpathy Builds Next — and What It Signals for ML Research

Karpathy’s stated next step goes well beyond a single GPU. In a follow-up post on X, he outlined a “SETI@home style” vision: collaborative agent networks where the goal is “not to emulate a single PhD student” but “to emulate a research community of them.” Thousands of consumer GPUs running coordinated experiments, sharing discoveries across nodes.

This raises a question nobody in ML hiring wants to confront: if a CEO can match a research team’s output on a real product overnight, what is the ROI of a five-year ML PhD specifically for running experiments? The answer isn’t obsolescence — it’s redefinition. The valuable skill shifts from running experiments to writing the program.md files that steer agents toward the right research questions.

Karpathy’s README puts it with characteristic bluntness: “One day, frontier AI research used to be done by meat computers. That era is long gone.” The same pattern is emerging in AI agents conducting autonomous scientific experiments in biology and chemistry — autoresearch is the ML-native version of that shift.

The genuine open question autoresearch raises: if program.md quality becomes the competitive differentiator in ML research, does that favor large labs — who can hire domain experts across every field — or individuals who have deep expertise in one specific problem? The tool democratizes execution. Whether it democratizes research outcomes depends entirely on who writes the best instructions.

Autoresearch doesn’t automate ML research. It cleaves the discipline in two — separating the part that required coding skill from the part that required domain knowledge — and hands the first part to an agent. That separation is more disruptive than any benchmark improvement, because it redefines who gets to participate.

Whether the SETI@home vision materializes or autoresearch stays a single-GPU tool, the repository’s 10,000+ stars in 48 hours already answered the demand question. The supply question — who writes the best program.md — is the one that matters now.