My second book on AI, Co-Existence, is out today (and you can get it here), 30 months after the release of Co-Intelligence in early 2024. Looking back across the gap between these books is a good way to get a sense of AI progress. And, thanks to AI, I can describe this progress with some quite lovely charts.
Making the charts is itself a good indicator of how far things have come. In the era of chatbots, the era of Co-Intelligence, I would have had to work back-and-forth with an AI in a chat window, guiding it through each new step. In the current era of agentic AI that I cover in Co-Existence, I delegated the work and let the AI handle it. In this case, I used Opus 5.5 in Claude Projects, which allows you to talk to a single “orchestrator” agent that then spins off additional agents as needed, giving them instructions and relaying information back to you. Getting the work done was straightforward. I gave the AI a couple sentences of suggestions and it did what would have taken me hours of work (all the research, all the graphing, all the layouts, all the images, all the citations…) in minutes. And since I didn’t have to do the work, I had the AI make many options for me. I picked some and gave handwavy reactions to others that would drive a human team mad (“make it better”). It even caught that I’d gotten the math wrong. I was going to call this post “28 months on the exponential.”
“This is the worst AI you will ever use”
I wrote this in my last book on AI, and it has continued to be literally true at every moment since. From early 2023 through early 2024, there was a single AI model that was undeniably the best in the world: GPT-4 from OpenAI. This was the AI model that most people remember from the first couple years of AI use, it was the one I thought about while writing Co-Intelligence, and it was the one that gave ChatGPT its status as the default AI for many people. Since then, twenty different AI models have held the lead at one time or another, all from Anthropic, Google, or OpenAI.
But the changing lead alone hides an even bigger change. A total of 180 AI models are now better than GPT-4 was at its release, many of which are open weights models, which anyone is free to use or modify. Nineteen of them can run on a modern smartphone.
We are living in an exponential
The data below draws on the METR Long Horizon Benchmark, which used to be the most famous chart in AI. The benchmark measured the amount of human work that an AI can do with at least a 50% chance of success from a single prompt. Gains on this benchmark have been startlingly regular. From the release of GPT-4 to the end of the METR measure, the amount of work AI could do doubled eight times. In 2024, an AI could do a four minute task. At the moment I turned in my draft of Co-Existence in April of 2026, an AI could do 17 hours of human work.
And then the benchmark ran out.
METR said they could no longer accurately measure the amount of work an AI could do using their current methods. Other studies suggest that number might be 17 weeks of work, or more.
And, even as ability has increased, prices have dropped. My old favorite AI benchmark, the Google Proof Question and Answer Exam, which gives the AI problems that regularly stump graduate students, used to cost 30 cents in AI usage to get a single correct answer. Now, it costs four hundredths of a cent to get the same result.
How humans work with AI is changing
Despite all the progress, the frontier of AI ability is still jagged, just as it was 30 months ago. Some areas have stayed stubbornly weak (AI is still quite a bad long-form writer), while others, like solving math problems, went from terrible to superhuman in the same period. And, as I discuss in the book, even where AI is excellent, small gaps create large needs for humans, at least for now.
But AI’s gains have been so large that how we work with it has to change. In Co-Intelligence, I wrote about centaur work (divided between AI and humans) and cyborg work (blended between AI and humans). Both keep you in the loop… but you can only be in so many loops at once. Now that we have real agents, I added a third way to work with AI: the conjurer. Conjurers summon agents when needed, assign them work, and evaluates what comes back. The charts in this post were made that way: I summoned agents, gave some general guidance, and I said “make it better” a lot. It is the most powerful way to use AI, and it carries new risks as we lose control over what, exactly, the AI is doing to get us those results.
The Otter Test has fallen
Regular readers will know that I have been testing AI image generation and movie tools with the prompt “Otter on an Airplane Using Wifi” for the past few years. This is what the best image generator, Midjourney, produced 30 months ago.
So today I asked Opus 5.5 the following (typos and all, prompting has become much less important as AIs get better): “I have a post showing the what happened over the past 30 months of AI in honor of my new book (published 30 months after the origina). i want you to make the ultimate otter on an airplane using wifi video. just something fantastic, i know you are creative, give it a go.” Using my subscriptions to various online services from Google and ElevenLabs, it produced the following. (Turn the sound on)
The video has a few flaws. But the AI figured out how to make it, from the script to the images to the concept to the music to the technical aspects of creating all of this, without any help from me. That is what has changed.
This is the least we will ever talk about AI
As a result of all of these gains, AI is only becoming a bigger issue in the world. This timeline of achievements and controversies since Co-Intelligence came out gives you some idea of how much is happening (I am leaving out hundreds of important events, but, even with the power of AI, we are all having trouble keeping up).
Of course, the speed of AI advancement is a problem for an author like me, which is why my new book does not try to keep up with the news or the latest model releases. Rather, it focuses on trends like the ones in this post: AI is improving exponentially but unevenly, and the way we use it is shifting from chatbots to agents that do real work. The book is about what that means for the humans who have to co-exist with these systems: where we still have an advantage (I argue it comes down to deep knowledge, wide knowledge, taste, and agency), when to hand work to AI, and when to avoid it entirely. If you understand the trends, the present (and near future) becomes much less surprising.
Co-Existence is out today. You can get it here, on a website Opus 5.5 redesigned. I told it to make it better.
