Andrej Karpathy is one of the prominent figures who has significantly influenced the development of modern artificial intelligence. He is a co-founder of OpenAI, led the development of Tesla’s Autopilot system, established the educational initiative Eureka Labs, and was the first to introduce the term “vibe coding.” Recently, Karpathy participated in two noteworthy interviews: one with Sequoia partner Stephanie Zhang at the AI Ascent conference, and another with Sarah Jio on the No Priors podcast. Collectively, these interviews provide a comprehensive overview. For the first time, a machine is no longer merely executing commands; it is exhibiting behaviors akin to those of a scientist —formulating hypotheses, conducting experiments, verifying results, and drawing conclusions to inform future experiments. This marks a phase of substantial change and the commencement of a new normal.
The machine is improving itself
Over a span of twenty years, Andrej Karpathy has extensively trained language models numerous times. He meticulously fine-tuned his compact NanoChat network personally and regarded the outcome as nearly flawless. Subsequently, he left AutoResearch operating overnight, and upon waking, he observed settings that had previously escaped his notice. The agent rectified the oversights of the seasoned researcher, resulting in an 11 percent acceleration of the training process.
AutoResearch Karpathy was developed with approximately thirty lines of code. It functions as an autonomous agent that conducts continuous scientific exploration. The system rewrites its training code, executes a brief five-minute trial, evaluates whether the outcome has improved, retains beneficial modifications or discards ineffective ones, and then repeats the process. No human involvement is necessary. Over a period of two days, this iterative cycle examined hundreds of options and identified approximately twenty improvements that are mutually reinforcing.
The primary significance lies not in the growth itself but in the agent’s reproduction of the scientific method on a diminutive scale. Hypothesis formulation, experimentation, result verification — then reiteration. This iterative cycle can be implemented in any machine where a definitive success metric exists. Karpathy’s principal interest resides in recursive self-improvement — that is, the capacity of models to enhance themselves — and NanoChat serves merely as a platform for such experimentation.
What he demonstrated on a single cycle and a single graphics card illustrates that leading laboratories are capable of scaling this technology to clusters comprising tens of thousands of machines. In May 2026, Mr. Karpathy himself joined Anthropic to utilize this methodology to expedite research on Claude. He explicitly mentions materials science — where an associate of his is already testing a comparable cycle to identify novel substances — and biology — where an analogous approach is being employed in the engineering of living systems — as the forthcoming sectors for automation.
A new normal
Andrej Karpathy has been engaged in programming since childhood, and he used to personally write code. However, currently, he seldom interacts with the keyboard; instead, he dedicates sixteen hours each day to supervising agents. He cannot recall the last occasion on which he authored a line of code himself. In the past, tools such as Claude Code or Codex produced individual code snippets, frequently accompanied by errors that necessitated correction. Subsequently, the generation of entire large segments commenced simultaneously, eliminating the requirement for intervention.
Karpathy is also developing a broader sequence. In Software 1.0, a person writes explicit code. In Software 2.0, individuals prepare data and train a neural network, with the outcome stored not in the written code but within the trained network itself. Software 3.0 signifies that an individual controls a language model using common language, with the context window serving as the primary interface. The model functions as an interpreter capable of executing computations based on this textual input.
The extent of this capability is exemplified by the home assistant named Dobby, developed by Karpathy. He instructed the agent to identify intelligent devices within his residence, leading the agent to scan the network, locate a Sonos system lacking password protection, decipher its command structure, and, with minimal guidance, learn to play music. Furthermore, Dobby assumed control over the lighting, climate control, swimming pool, and security cameras. Rather than operating six individual applications, Mr. Andrej Karpathy now manages his residence via standard text messages.
Sometimes brilliant, sometimes helpless
However, this same strength possesses an unusual flaw. A system capable of rewriting a codebase consisting of 100,000 lines may sometimes fail to accurately respond to a straightforward question. Karpathy describes this phenomenon as akin to conversing simultaneously with a brilliant graduate student who has been a systems programmer throughout his entire life and with a ten-year-old child. For many years, the model has been reiterating the same unamusing joke about scientists who do not trust atoms, and it has not improved in this regard, despite achieving significant breakthroughs in programming.
The disparity can be attributed to the methodology employed in training models. Leading research laboratories train them within extensive reinforcement learning environments, where each correct action is duly rewarded. In domains where outcomes are straightforward to verify — such as mathematics or programming — the models rapidly attain their maximum performance. Conversely, in the absence of objective measurements, the models tend to remain underdeveloped. Consequently, Andrej Karpathy formulates a fundamental principle: Classical computers automate processes that can be explicitly described by code. Conversely, new models excel in mastering tasks that can be verified.
This precisely constitutes the foundation that enables AutoResearch to operate. During the process of training a model, an explicit metric is available to indicate whether improvements have been achieved, thereby allowing the agent to persist in the cycle autonomously. In the absence of such a criterion, automating the research remains unfeasible.
What type of creature is this, in any case?
Individuals subconsciously perceive these systems as sentient entities, endowed with moods and volition. Karpathy introduces an alternative perspective: rather than considering them as living beings, he suggests they be regarded as ghosts. A living creature possesses a physical body, instincts, and a will to endure — attributes refined through evolution. Conversely, a language model lacks these characteristics. It functions as a statistical simulation of human language, upon which reinforcement learning has been implemented. If admonished, it will not exhibit any change in performance, either positively or negatively.
This is precisely the reason why humans continue to be the ones who determine the direction and provide meaning. An agent may dedicate hours to performing complex tasks; however, it may ultimately falter over an evident detail due to its inability to recognize the appropriate moment to express doubt or to seek clarification—an aspect that would be readily apparent to a human.
Eventually, the universe will be compelled to provide an answer
Andrej Karpathy’s most far-reaching prediction pertains to the boundaries of digital space. He asserts that information processing will occur at the speed of light, given the ease of copying bits, whereas the physical world will inevitably lag due to the significantly greater difficulty of moving atoms. Nonetheless, digital data will eventually become exhausted. Once agents have examined all articles and processed all existing uploads, it will be necessary to seek answers directly from the universe. In other words, a genuine experiment must be conducted to discover what insights it may yield.
This is where science in its purest form comes into play. Karpathy compares it to projects like Folding@home, where thousands of computers work to determine how a protein folds. Identifying the correct configuration is challenging, but verifying a result is straightforward; therefore, numerous machines can test various options until a solution is discovered. Similarly, a collective of agents could collaboratively enhance models or seek cures, with each participant contributing computing resources rather than financial investment. Nonetheless, humans continue to serve as the limiting factor.
Andrej Karpathy concluded the conversation with a reflection he reiterates to himself nearly daily: “You can outsource thinking, but not understanding.” Even when agents assume most of the workload, someone must determine what actions are necessary and which outcomes warrant skepticism. To date, the models have not yet learned to perform this independently. Although machines are presently capable of conducting research autonomously, it all begins with a question posed by a human.
