Let's start with a question. What happens when our ability to discover new things moves beyond our ability to understand them?
But first, let's take a step back. I think it's fair to say that science has typically had a bit of distance between discovery and explanation. We find something interesting, and then we go and try to figure out why. And there's an expectation that understanding will eventually catch up. It's almost like "knowledge causality" where the two are linked.
Artificial intelligence (AI) may be complicating or even breaking this linkage in ways we have not fully considered. I think there are two different problems we need to address here. The first concerns the extraordinary computational capabilities of AI. The second is a little stranger and, to me, much more interesting. It's our ability to understand what all that computation actually means, particularly in the context of human cognitive capabilities.
The Limits of Computation
The complexity of AI is often hidden behind the simplicity of the user experience. Large language models (LLMs) operate in an almost "ethereal complexity" of numbers and dimensions. The math behind these systems is well understood, but the enormous number of interacting relationships makes it difficult to explain precisely, well, what the heck is actually going on.
This is the ongoing problem of AI interpretability. We know the way that AI is constructed, and we're even making progress in understanding their inner workings. But the sheer complexity of their computations may exceed what any human could reasonably follow.
There's something important here. Just because we can't follow every calculation doesn't mean we can't understand the result. We use computers to solve problems that would take us lifetimes to crunch the numbers ourselves. The answer, however, arrives in familiar and palatable terms that we can work with.
So far, so good. But there's another boundary ahead—one that may be much more difficult to cross.
The Limits of Understanding
What if an answer becomes incomprehensible to humans?
Imagine AI discovering relationships across thousands of interactions. The relationships may be mathematically valid and extraordinarily useful. But the discovery itself might exist in a form that doesn't really translate into anything a human can understand or conceptualize.
Sidebar: It might be fun to get a sense of this complexity with a few back-of-the-envelope calculations. GPT-3, an older model, contained 175 billion parameters and operated across 12,288 dimensions. Meta's Llama 3.1 expanded that to 405 billion parameters and 16,384 dimensions. Don't try to understand this explicitly; just try to get a sense of the profound complexity of today's LLMs.
So, we can describe the math. But making sense of what all those relationships actually represent is another matter entirely. We might be able to test the result and establish that it works. Or maybe another AI could even use that discovery to develop something new. But across this sea of complexity, humans could remain unable to understand the "first-principle" relationships that give the answer its meaning and utility.
A Strange New World
Humans have always used mathematics to explore things we cannot visualize. Complexity alone doesn't make something unknowable. But AI establishes the strange possibility that some discoveries may remain beyond our ability to have a meaningful explanation. And that holds true even when we can demonstrate their usefulness.
So, here's the key insight: AI intelligence may allow the advancement of knowledge to become independent of human understanding.
Think about that. One AI makes a discovery that humans cannot adequately explain. And then another AI builds upon it, even building on relationships that remain outside our comprehension. A third develops something new from those earlier discoveries. The process continues, and knowledge advances without necessarily passing through the human mind in any meaningful or interpretative way.
Artificial Intelligence Essential Reads
The Black Boxing of Reality
Simply put, the black box is a system we can use without really understanding the inner workings—we know what goes in, we observe what comes out. But imagine that construct moving beyond the computer itself. AI-generated discoveries become the foundations for new discoveries, which become the foundations for still more discoveries.
We could eventually inhabit a world whose underlying structure and organization are built upon discoveries that we can't fully translate into our understanding. AI might work exceptionally well. We could test them, use them, and even improve them. Yet important parts of their output might remain intellectually inaccessible to us.
There's a psychological consequence here that I find fascinating and critical. We like to understand things, particularly the things that shape our lives. Understanding allows us to question an answer, challenge an assumption, and frolic in the world of discovery. The Nobel Laureate and acclaimed physicist Richard Feynman captures this perfectly in the title of one of his books: The Pleasure of Finding Things Out.
Push on this a little bit, and here's what we might get. We get progress that no longer requires understanding. We become comfortable accepting results simply because they work. And our confidence could become detached from understanding. And when that happens, our curiosity may begin to change as well. It's a precarious spiral predicated on the simple idea of "Why bother?" Why bother to understand something when the next (AI) discovery is already waiting?
I think it's important to recognize that this isn't a pessimistic view of AI. AI may ultimately help us explain things we never could have understood on our own. But there's another possibility that deserves some reflection. We may become extraordinarily capable of creating things while becoming less capable of understanding what we've created. And that may be humanity's greatest paradox.
Science has helped cast a light on the mysteries of our world. Now we may be developing intelligence that allows us to create mysteries of our own—building upon them and into the fabric of everyday life.
The black box may eventually become the world itself.