Summary: AI's rapid advancement is caused by three factors: raw compute increases, algorithmic efficiency improvements, and "unhobbling" processes. In the current super-accelerated decade, we enjoy a 10,000x AI performance boost every four years, making it likely that AI will exceed top human experts by 2027. There are good arguments that AI progress past 2027 could be slower, but also that it could be even faster.
Hold onto your GPUs, folks! AI has gone from drooling toddler to teen genius faster than you can say “singularity.” This article explains the silicon-powered rollercoaster of AI progress, where Moore's Law looks like a lazy snail. Spoiler alert: Things are about to get weird!
The best explanation I have seen of long-term trends in AI and the likely implications for the next decade’s worth of AI progress was recently published by Leopold Aschenbrenner.
Aschenbrenner’s broad conclusions are very similar to what I wrote in my piece Compute Creates Usability, with two exceptions:
Aschenbrenner goes into much more detail, with rich charts to substantiate the points. I strongly recommend reading his (very long) analysis if you have time. He also proceeds to analyze several broader future implications I skipped, such as the relationship between the world’s leading superpowers.
I added a component to my analysis of the growing impact of AI that Aschenbrenner skipped (or missed): the usability of AI tools. When (or, sadly, if) more UX people start working on AI tools, their usability will improve such that users get better results from the same level of underlying AI capability. I include a usability analysis because I view the most promising future as one of human-AI symbiosis, where the two work together. Thus, AI alone is less important than the abilities of humans and AI to collaborate.
Aschenbrenner distinguishes between raw compute and effective compute, with the latter being what counts. Raw compute is the progress in the number of FLOPS (or other count of operations per second) produced by the available computer chips. This is currently progressing at a rate of slightly more than 3x per year.
Effective compute also considers the transformation of these raw GPU cycles into AI results. Improvements in the underlying AI algorithms happen at about that same scale, or about 3x per year. In other words, next year, we’ll only need 1/3 of the computer chips to achieve the same level of AI we’re enjoying right now. (Of course, instead, we’ll have more chips and better AI next year.)
Multiply the two, and effective compute increases by about 10x, or an order of magnitude, every year.
In 4 years, effective AI compute increases by a factor of 10,000. This is exactly what happened between 2019, when AI could barely compose a complete sentence, and 2023, when GPT-4 could ace high school exams. In 4 years, AI’s intellectual prowess “aged” by about 14 years in human terms from being worse than a 4-year-old human to being better than most 18-year-old humans. 3.5x the “childhood development” pace seen in biological intelligence.
The jump in AI quality from 2019 (when GPT-2 could barely produce a complete sentence) to 2023 (ChatGPT 4, which is as smart as a good high school graduate) is expected to be equaled by the jump from 2023 to 2027, when AI will become as skilled as the best human experts. (Ideogram)
The expectation is that the jump in the subsequent 4 years (from 2023 to 2027) will be of the same magnitude, i.e., another factor 10,000 in effective compute, which will bring AI to the level of the top human experts in most intellectual fields.
Aschenbrenner discusses another series of continuous advances in AI, which he calls “unhobbling.” This is difficult to summarize, but basically involves removing various restrictions on the AI’s ability to deal with the world. One unhobbling that has already partly happened is to allow for big enough context windows to allow AI to chew on meaningfully large amounts of user data. An unhobbling he expects during the second 4-year period (where we’re currently in Year 1 of 4) is a full carry-over of history so that the AI can work on longer-term projects and not just small case-by-base actions. Even with the same amount of base intelligence, the more we unhobble AI, the more useful it will be for solving practical problems (as opposed to exam quizzes, which is the main way AI is currently evaluated).
Aschenbrenner’s 3 ways of improving AI functionality: (1) More raw compute cycles, from better AI chips and bigger data centers. (2) More efficient algorithms to translate compute cycles into effective AI performance. (3) Getting better AI features from that AI performance by breaking the shackles that prevent AI from connecting to the real world. My article adds a fourth factor (not shown): improving usability to make those AI features more useful. (Leonardo)
Self-Improving AI May Accelerate Even Faster
Where things start getting interesting is Aschenbrenner’s speculations about the next 4-year period, after we arrive at top-human performance in 2027. He acknowledges that top theoretical performance won’t necessarily translate into top practical performance in all fields. For example, healthcare and education may be held back for political purposes, since they are subject to substantial regulatory capture by the legacy humans who currently do those jobs (doctors and teachers).
But AI engineering is one field where Aschenbrenner expects full utilization of top-performing AI. We will have the equivalent of a million top AI researchers working on making AI even better. This is about a thousand times more than the top AI researchers currently working on advancing the leading frontier models. This hugely expanded effort will likely lead to substantial advances in algorithmic efficiency (thus upscaling effective compute, even if raw compute advances are expected to slow down) and in unhobbling.
Once AI becomes good enough to improve itself, we’ll start to experience truly fast-moving advances. The current situation is nothing compared to the decade after 2027. (For the time after 2037, maybe we’ll get the singularity, or maybe we’ll encounter some unforeseen difficulties that slow down subsequent advances.)
We’re currently about 4 years into a decade with unusual advances in raw compute that can’t be expected to continue. The same is true for human-invented algorithmic efficiency advances and unhobbling advances, but as mentioned, Aschenbrenner does expect the non-hardware improvements to continue to accelerate due to the shift to AI-produced innovation.
One-Time Super-Acceleration Causes
The reason we currently have unsustainable acceleration is that truly big investments in AI didn’t start until recently. Raw AI compute currently accelerates far faster than standard computing, because we’re suddenly willing to invest hundreds of billions of dollars in building AI data centers for training and inference compute. Aschenbrenner expects trillion-dollar data centers toward the end of the 2020s, but we’re unlikely to go much further than investing at the level of tens of trillions of dollars in AI hardware. (The world’s entire GDP is USD $140 T in PPP, and even in the likely case that AI doubles world GDP, that will leave maybe $50 T for AI investments, since most of our added wealth should be spent on improving humanity’s standard of living.)
AI raw compute also accelerates faster than old-school computers because companies like NVIDIA have started designing AI-specific chips. For the first several generations of custom-designed silicon, domain-specific chips can wring out a lot of efficiencies compared to general-purpose silicon. But eventually, advances in AI-specific chip-design should slow to approximately the pace characterized by Moore’s Law. (Still improving every year, but not by nearly as fast as we enjoyed during the decade 2019-2029.)
On the algorithm side, only very few of the world’s top geeks used to work on AI, since it was considered to be a perennial loser topic. Now, the situation is the opposite. Anybody who’s any good in technology now wants to work on AI, with the losers left to work on non-AI projects. However, this shift from AI attracting only a few top talents to attracting all the top talents can only happen once. This change is also happening in the current decade, so we’re enjoying an additional acceleration in algorithmic efficiency from putting better talent on the job.
AI is accelerating at an unprecedented pace during the decade we’re about halfway through. Later, unsustainable superior improvements in raw compute and human-discovered algorithmic efficiencies will slow to match the traditional pace of general computing. However, AI itself may super-accelerate additional discoveries in algorithmic efficiency and thus keep up the acceleration for another decade or more. (Midjourney)
Implications for UX
First of all, for UX professionals: you urgently need to get an AI-focused job to be relevant for the future. AI is the only thing that matters because of the enormous changes over the upcoming decade. We’re in the early days of an exponential growth curve of AI capabilities. Change may seem fast, but if you know exponential growth curves, you know how much faster it’ll get. By then, it’ll be too late to get on the bandwagon.
Second, for UI design: expect AI capabilities to be so enormously better in a few years that anything you design now will have to change. You should still give your customers integrated AI capabilities in their user experience now, because AI is already good enough to be useful for many tasks. But UX strategy and UI prototyping should assume you will soon package a team of 200 Ph.D.-level experts in a lunchbox.
By 2027, AI will be like bringing a lunchbox to work filled with 200 Ph.D.s from all disciplines. (Leonardo — sorry, it could not draw all 200 Ph.D.s within the lunchbox.)
The point to remember is that exponential change seems slow in the beginning, because the amount of linear change is small, even if the relative change is large. But as the baseline grows and grows, that same percentage of change per time period becomes a larger and larger linear change.
An anecdote to illustrate: In March 2020, I was scheduled to speak in London. Before leaving on the trip, I read the news that there were 10 Covid-19 cases in London. I thought that 10 sick people was nothing, given the size of London — the probability of being on the same carriage in the Tube as one of those 10 people was essentially zero. A few days later, I read that there were now 20 Covid cases in London, but my thinking remained the same. A few more days passed, and I read that 40 people had come down with Covid in London. Still the same point about the small likelihood of running into one of those 40 people in the big city, but the real analysis was that the condition was spreading exponentially. Sure enough, by the time my conference was supposed to have happened, London was shut down due to Covid.
Exponential change looks like a hockey stick: at first, the experienced change isn’t that big, but then it takes off, and that’s what is about to happen with AI. For a demonstration, watch my video showing the changes in Midjourney’s image quality from 2022 to 2024. Each individual change isn’t that big, but they accumulate. And those examples were from the flat part of the curve!
Exponential growth — as we’re experiencing during the 2020s for effective AI functionality — starts out small. Even at a high rate of change, the next few observed instances still look small. That’s where we are now with AI. Later, even if the rate of change remains the same, the observed size becomes big (AI in 2027) and then humongous (AI in 2030). (Ideogram)
Comments