Andrej Karpathy — “We’re summoning ghosts, not building animals”

Karpathy identifies the bottlenecks impeding agent development, notably the absence of continual learning and robust general intelligence. Without the ability to remember and learn continuously, agents reset at the start of each session, preventing genuine adaptability and growth over time. He foresees solving these challenges as a process spanning about a decade, grounded in his long-standing experience in AI and observing how past predictions have unfolded more slowly than initially hoped. This timeline accounts for the incremental work needed on fundamental cognitive capabilities, not just scaling existing models or applying surface-level fixes.

Historical Shifts in AI

Reflecting on his nearly two decades in AI, Karpathy recounts the seismic changes that have shaped the field: the early niche work on neural networks, the breakthrough with AlexNet that reoriented AI toward deep learning, and the later advent of reinforcement learning with Atari games. He critiques some prior emphases, such as heavily focusing on reinforcement learning in gaming environments, which he believes was a misstep that detracted from direct progress on versatile, real-world agents. The excessive reward sparsity in such environments limited learning efficiency and required massive computational effort without achieving generalized intelligence.

Karpathy underscores his own efforts at OpenAI during that time, where he sought to develop agents capable of meaningful interaction in real digital environments—using keyboard and mouse to manage webpages and execute workflows. These ambitions were too early and proved impractical at that stage, illustrating how some attempts to build full agents before foundational capabilities were in place ultimately faltered. In contrast, modern agents benefit from pre-trained large language models that provide massive representational power, paving the way to build more competent and flexible systems atop this foundation.

"Summoning Ghosts, Not Building Animals"

A recurring theme in Karpathy's analysis is the distinction between building AI that mimics human or animal brains directly and creating what he terms "ghosts" or "spirits." He contrasts his views with perspectives like Richard Sutton's which posit that replicating animal-like intelligence from sensory inputs alone is the target. Karpathy cautions that the evolutionary process which fashioned animal intelligence is fundamentally different from how AI models are trained: evolution encoded learning algorithms via DNA and a massive outer loop, whereas neural networks today are trained by imitating vast internet text.

He points out that animals inherit vast amounts of hardwired knowledge and instincts evolved over millions of years, whereas current AI agents are constructed by distilling publicly available data, yielding digital entities that reflect human knowledge but lack the innate scaffolding animals possess. Karpathy regards large-scale pre-training as a crude proxy for evolution—a way to assemble starting knowledge and cognition—which is why these AI systems behave like "ethereal spirits" rather than complete biological agents. This distinction explains some of the limitations of current agents and why achieving lifetime continual learning akin to animals remains elusive.

Continual Learning Challenges

Karpathy eloquently discusses the mechanism behind in-context learning—the phenomenon where models appear to learn and adapt during conversations without weight updates. Noting that in-context learning is essentially pattern completion within the prompt window, he suggests it may internally parallel a kind of rapid gradient descent, allowing the model to adapt its reasoning dynamically. Despite this, he stresses a huge disparity between the limited information compression in pre-training weights and the far more direct availability of information in context windows, likening in-context tokens to working memory and model weights to long-term hazy recollections.

However, this kind of ephemeral learning falls short of full continual learning, which would require models to absorb, distill, and integrate experiences over long periods into their permanent knowledge without forgetting. Karpathy notes that humans compress daily experiences into their brains during sleep and reflection—processes currently absent in large language models. The absence of an analogue to sleep or dreaming, where information is reprocessed and consolidated, limits AI's ability to truly learn from extended interactions, presenting a significant hurdle for realizing agents that improve organically over time.

Limitations of Reinforcement Learning

In addressing the shortcomings of reinforcement learning, Karpathy offers a critical perspective that it is often misapplied or misunderstood in the context of achieving intelligence. He explains that traditional RL broadcasts a sparse reward signal—like success or failure at the end of a trajectory—across all actions, indiscriminately reinforcing the entire sequence, even the incorrect steps. This noisy and imprecise feedback contrasts sharply with how humans learn, which involves deliberate reflection on which parts of a process were effective or mistakes.

Karpathy highlights the difficulty of implementing process-based supervision—giving rewards or feedback at multiple points along a task rather than a single final reward. Attempts to overcome this with AI "judges" to provide intermediate reward signals often run into adversarial gaming by models, producing nonsensical outputs that ironically score highly. This intrinsic vulnerability means the reinforcement learning loop remains trapped in noisy gradients and simplistic credit assignment, making breakthroughs in fine-grained process supervision pivotal but technically challenging next steps in advancing AI capabilities.

The Architectural Future of AI Models

Karpathy shares his intuition on the future evolution of AI architectures, predicting that while there will be significant algorithmic improvements, the fundamental structure—a large neural network trained with gradient descent—will remain broadly intact over the next decade. He points to historical progress in deep learning, noting that major leaps have generally been the result of simultaneous improvements in hardware, software kernels, data quantity, architectural motifs, and optimization techniques, rather than any single dominating breakthrough.

He expects developments like more sophisticated sparse attention mechanisms and multimodal training to enhance model capacity and efficiency. Reflecting on his experiments reproducing early convolutional networks from 1989, Karpathy underscores that many advances are incremental and complementary. Despite the proliferation of enormous trillion-parameter models, he envisions cognitive cores capable of meaningful intelligence to be potentially much smaller in the future, provided that the training data quality is drastically improved and the redundant memorization of internet noise drastically reduced.

Practical AI Engineering

Turning to his recent work on nanochat, Karpathy provides an insider's perspective on the practical challenges of building large-scale AI systems from scratch. Though AI autocomplete and code-generation tools can assist with boilerplate or familiar code snippets, they struggle with novel, architecturally complex, and heavily customized codebases like nanochat. The models often misunderstand unique design decisions or push standard patterns that do not fit the developer's intentions, resulting in bloated or incorrect code suggestions that require thorough human oversight and cleanup.

Karpathy emphasizes that the cognitive limitations of current models make them ill-suited to architect and integrate new features coherently at the system level. While powerful models such as GPT-5 Pro can be used as oracles to answer specific questions or debug small fragments, the creative and precise orchestration of a novel AI system still requires expert human involvement. This experience tempers overly optimistic narratives of AI accelerating AI engineering to an explosive scale, suggesting instead a more measured augmentation similar to improved compilers or tooling rather than fully autonomous AI scientists.

AI Adoption

Karpathy situates current AI progress within economic and societal contexts, noting that the most tangible automation so far occurs in knowledge work with structured inputs and outputs, such as coding. He highlights the challenges of deploying AI broadly in other knowledge domains like call centers or radiology, where workflows are messier, safety requirements are stringent, and human judgment remains indispensable. While AI can automate routine portions, full replacement is unlikely to happen abruptly; more probable is a gradual "autonomy slider" where AIs handle increasing subsets of tasks under human supervision.

Drawing parallels with self-driving cars, he discusses how expensive capex, latency constraints, and liability concerns slow adoption despite technological feasibility. The analogy extends to labor market dynamics where incremental automation shifts human roles rather than removes them wholesale. Karpathy predicts a slow diffusion of AI into society consistent with historical automation trends—rather than sharp discontinuities or runaway intelligence explosions—emphasizing the complex interplay of technology, regulation, and economic incentives in shaping AI's real-world footprint.

The Intelligence Explosion

Throughout the conversation, Karpathy expresses skepticism about the notion of a sudden intelligence explosion driven by a uniquely capable AGI. Instead, he portrays AI progress as part of a continuous extension of computing and productivity improvements that have been unfolding for centuries. From the Industrial Revolution through the rise of computing and beyond, economic growth trends fit an underlying exponential curve sustained by recursive technological advancements and labor automation.

He nonetheless acknowledges that AI, by enhancing human-like cognition and enabling a large population of autonomous agents, could accelerate certain aspects of progress. Yet he anticipates this acceleration will manifest as an intensification of ongoing trends rather than a discrete singularity event. While superintelligence would appear qualitatively different and potentially alien, Karpathy foresees its societal impact as a complex, emergent competition among autonomous systems, involving loss of human oversight but not immediate catastrophic changes.

Self-Driving Car Development

Karpathy recounts his tenure leading Tesla's self-driving efforts, highlighting why deploying such systems at scale has taken decades despite early impressive demos. He uses self-driving as a vivid example of the "march of nines"—each increase in reliability by a factor of ten requires roughly the same enormous effort. The critical safety domain amplifies the testing and validation burden, necessitating near-perfect performance levels impossible to shortcut. Early Waymo demos looked promising but posed massive economic and operational challenges in real-world environments.

Tesla's approach aimed for more scalable solutions and progressive deployment, but the technology is far from finished. Karpathy reveals that despite removal of human drivers in some contexts, remote human operators still support car fleets, adding complexity and cost unseen by casual observers. The analogy between self-driving and reliable coding agents is telling: both have low tolerance for catastrophic errors and require immense unseen engineering toil behind the scenes. This long, painstaking timeline tempers expectations of rapid AI deployment and cautions against conflating proof-of-concept demos with mature production readiness.

Education as the Key to Human Empowerment Amid AI Progress

Concluding with a forward-looking vision, Karpathy shares his commitment to education through his project Eureka, aspiring to create a "Starfleet Academy" for frontier knowledge that empowers humans to thrive alongside evolving AI technologies. He stresses that while AI labs focus on the automation frontier, the unique human element remains critical: ensuring people understand, control, and can influence these powerful systems. Learning that is appropriately scaffolded, adaptive, and motivating can unlock latent human potential at a scale never before possible.

Karpathy draws on his own experiences with successful one-on-one tutoring—highlighting its ability to pinpoint knowledge gaps, provide perfectly calibrated challenges, and sustain motivation—as a gold standard AI-driven education has yet to achieve. He envisions progress as moving through phases: initially human experts assisted by AI tutors, gradually transitioning toward AI systems taking on more substantive instructional roles as their capabilities mature.