Andrej Karpathy on AGI, Software 3.0, and Why the Next Decade Changes Everything

Andrej Karpathy is one of the few people who has built AI systems at three completely different scales. He co-founded OpenAI. He led Autopilot at Tesla. He left both to teach neural networks to the world through his YouTube channel and Eureka Labs. When he sits down with Dwarkesh Patel for a long conversation, you pay attention.

The interview covered a lot of ground, but two ideas kept coming back. The first is that AGI is probably about a decade away. The second is that the way we think about software itself needs to change, and Karpathy has a name for where we are now: Software 3.0.

What Karpathy Means by Software 3.0

The framing is worth understanding on its own terms because it changes how you think about what programmers do.

Software 1.0 was explicit instructions. A programmer sat down and wrote code telling a computer exactly what to do, step by step. Every behavior was specified in advance.

Software 2.0 was the neural network era. Instead of writing rules, you defined a loss function and let gradient descent find the weights. The "code" was learned from data rather than written by hand. This is what made ImageNet, AlphaGo, and GPT-2 possible.

Software 3.0 is the era we are in now. The program is written in English. Or Hindi. Or Japanese. You describe what you want, and a large language model executes it. The LLM is the runtime, and natural language is the programming language.

This sounds abstract until you realize what it means practically. Karpathy argues that LLMs are not tools in the traditional sense. They are a new kind of computer, one that runs programs written in human language rather than in Python or C. When you write a detailed prompt to get Claude or GPT-4o to do something complex, you are programming. The fact that you are doing it in sentences rather than syntax does not make it less real.

The implications for the profession are significant. Karpathy is not saying programmers become obsolete. He is saying the nature of programming expands. More people can program because the syntax barrier is gone. The bottleneck shifts from "can you write code" to "can you specify what you want precisely enough that the system can execute it."

His AGI Timeline

Karpathy is careful with the word AGI because its definition is contested. In the interview, he uses it to mean something roughly like: a system that can do any cognitive task a human can do, at human level or above, given the same inputs and time.

His estimate is roughly ten years. He is not committing to a specific date, and he is explicit that these predictions are hard. But he thinks the trajectory is clear enough to believe we will get there within the decade.

The reasoning is empirical rather than theoretical. LLMs have improved at a consistent rate. Each generation does things the previous generation could not. The gap between what GPT-2 could do and what GPT-4o can do is enormous, and that gap emerged in roughly four years. Karpathy does not think the scaling will suddenly stop. He thinks compute will continue to increase, algorithms will continue to improve, and the combination will get us to something qualitatively different from what we have today.

He also distinguishes between narrow AGI (being better than humans at specific tasks) and general AGI (being better at the full range of cognitive tasks). He thinks we already have narrow AGI in many domains. Writing at a high level, coding in most languages, answering factual questions, analyzing images. The remaining challenge is robustness, reasoning under genuine uncertainty, long-horizon planning, and physical grounding.

The Memory and Context Problem

One thing Karpathy highlights that often gets missed in AGI conversations is the memory problem. Current LLMs have a context window. They know what is in the conversation. They do not accumulate experience the way humans do. A model that finishes a conversation forgets everything unless it is explicitly given tools to store and retrieve information.

Human intelligence is partly constituted by accumulated experience. A doctor who has seen thousands of patients develops intuitions that cannot be reduced to textbook knowledge. A programmer who has debugged hundreds of systems develops pattern recognition that is hard to articulate. Current LLMs do not have this.

Karpathy thinks solving memory and persistent learning over time is one of the remaining hard problems. Context windows have grown dramatically, and retrieval-augmented generation helps, but the deeper issue of how a system learns from experience without catastrophic forgetting is not fully solved.

What This Means for Developers

The Software 3.0 framing has a practical implication that Karpathy makes explicit. If LLMs are the new computers, then the developers who thrive will be the ones who learn to work with them fluently. Not just as autocomplete, but as partners in building systems.

He uses AI tools constantly. He thinks developers who resist using AI coding assistants are making a professional mistake, not a principled stand. The question is not whether AI will be part of your workflow but whether you will learn to use it well enough to be productive.

He also thinks the education system needs to change in response. A lot of what computer science programs teach is low-level knowledge that LLMs can now handle. The curriculum should shift toward judgment, system design, and the ability to evaluate whether a generated system is doing what it is supposed to do.

Why the Karpathy Perspective Matters

He is not a hype machine. He left OpenAI at a moment when he could have stayed and become wealthy purely through equity. He made videos explaining backpropagation from scratch because he thinks education is genuinely important. When he says AGI is a decade away, he is not trying to pump a stock.

His view on Software 3.0 is also grounded in something he has actually built. He ran the team that made Tesla's Autopilot work at scale, using learned systems rather than hand-coded rules. He has lived the Software 2.0 transition from the inside. The Software 3.0 framework is an extension of thinking he has been developing for years.

The decade he is talking about is going to be strange for everyone in technology. The tools will keep getting better at a pace that is hard to internalize. The developers who read the Karpathy interview carefully and take the Software 3.0 frame seriously will be better positioned than those who treat LLMs as a fancy search engine.

The core message is simple enough: the computer that runs on English is here, it is going to keep improving, and the people who learn to program it will shape what the next ten years look like.