
The term “singularity” is often used to describe a hypothetical moment when artificial intelligence surpasses human intelligence and then begins to improve itself at an uncontrollable rate. More simply, people often use the term “singularity” to refer to the moment when AI becomes conscious, like Skynet becoming self-aware in the movie Terminator. But the notion of a “technological singularity” originated as a broader metaphor for a point beyond which technological change becomes so fast that the future is hard for humans to predict.
In 1958, mathematician Stanisław Ulam recalled a conversation with physicist John von Neumann about accelerating technological progress reaching “some essential singularity” in human affairs. At that point, the term didn’t refer to a theory of machine consciousness. But in 1993, mathematician and science-fiction writer Vernor Vinge made the concept influential in AI circles. In his book The Coming Technological Singularity, Vinge argued that the creation of greater-than-human intelligence could cause a runaway change in civilization, after which our ability to forecast events would fail.
The futurist Ray Kurzweil later popularized the term, especially through his 2005 book The Singularity Is Near. In that work, he gave “singularity” a more specific meaning, claiming that humans would merge with increasingly powerful nonbiological intelligence by 2045. So while many of us are worried today about the dangers of AI, and some experts even believe that the “AI singularity” is already here, Kurzweil’s idea is a lot more radical than that. He doesn’t just postulate that machines will develop the same abilities as human brains. Instead, he says that AI-driven machines will merge with our brains and become one with them. And in Kurzweil’s techo-optimistic view, that’s a good thing.
In 2005, that might have seemed a little farfetched. But given recent advances in neural prostheses and brain-computer interfaces—and of course in AI—Kurzweil’s prediction about the singularity seems much more plausible now. Still, some aspects of his cybernetic singularity remain counterintuitive, if not entirely hard to believe. Just how, exactly, does a machine merge with a human? Just how does human consciousness get uploaded into the cloud? Just who would “I” be if I’m an immortal digital copy of myself living in a nonbiological body?
In the 2024 revision of his book, retitled The Singularity Is Nearer, Kurzweil addresses those and many other similar questions. He emphasizes that the transfer of human consciousness from our bodies into nonbiological systems won’t happen all at once, or in just one way. In fact, the transfer’s already begun, in the form of digital simulations of our appearance and ways of thinking and speaking. Always the optimist, Kurzweil sees the upside of our contemporary deepfake problem.
As he puts it, “We are already creating through our digital activities enormously rich records of how we think and what we feel, and during this decade our technologies for recording, storing, and organizing this information will advance rapidly. As we approach the end of the 2020s, we will animate this data as nonbiological simulations that are highly realistic re-creations of humans with specific personalities.”
The process of transferring humans to digital replicas is also already happening in the form of devices like cochlear implants that allow deaf people to hear, systems that let people with paralysis control computers, and newer implants such as Neuralink. Kurzweil envisions increasingly robust forms of these and other artificial organs that will soon vastly improve our natural levels of strength and longevity. At a certain point, these different prosthetics and simulations will grow in sophistication and be combined into robotic versions of ourselves that will be so convincing that they’re indistinguishable from the original you.
Kurzweil borrows the name for this step in our evolution toward cybernetic singularity from Phillip K. Dick’s 1968 science fiction classic Do Androids Dream of Electric Sheep?—later brought to the big screen as Blade Runner. “Eventually replicants may even be housed in cybernetically augmented biological bodies grown from the DNA of the original person,” he writes.
And here’s where the question of who’s who and what’s what gets a little complicated. Because if a simulation of you is actually made of your DNA and contains all of your memories and personality traits, who’s to say it’s not really you? Kurzweil calls this kind of whole-person replicant a You 2. “How much would it matter whether a replicant was created through AI and data mining, as opposed to a fully uploaded You 2 mind from someone’s living brain?” he asks, although the implication is clear. A convincing deepfake is one thing, but a living copy imbued with every trait of the original person is another.
And Kurzweil points out that this brings up some head-spinning questions about You 2 replicants.
“Are they to be considered people with full human and civil rights (such as the rights to vote and enter into contracts)? Are they responsible for contracts signed or crimes previously committed by the person they are replicating? Can they take credit for the work or social contributions of the person they are replacing? Do you have to remarry your late husband or wife who comes back as a replicant? Will replicants be ostracized or face discrimination? Under what conditions should the creation of replicants be restricted or banned?”
The answers to these and similar questions have been the fodder of popular science fiction for decades—so they’re hardly unfamiliar, despite remaining entirely too weird for comfort.
But most importantly, all of these advances will culminate in a revolution led by nanobots. Currently, nanobots are used mainly for drug delivery, but for Kurzweil, they’re the key to our ultimate cybernetic evolution into nonbiological humans. Most importantly, they’ll make our brains millions of times more powerful than they are now. As he writes, “In the 2040s and 2050s, we will rebuild our bodies and brains to go vastly beyond what our biology is capable of, including their backup and survival. As nanotechnology takes off, we will be able to produce an optimized body at will: we’ll be able to run much faster and longer, swim and breathe under the ocean like fish, and even give ourselves working wings if we want them. We will think millions of times faster, but most importantly, we will not be dependent on the survival of any of our bodies for our selves to survive.”
Naturally, there are some potential downsides to all of this, and in Kurzweil’s vision, the most significant danger is that nanobots could go rogue and begin reproducing out of control. They could in fact reproduce so rapidly that they’d absorb literally all of the organic material on the surface of the Earth, reducing everything to piles of goop.
He writes, “The most commonly discussed worst-case scenario is the potential creation of “gray goo”—self-replicating machines that consume carbon-based matter and turn it into more self-replicating machines. This process could lead to a runaway chain reaction, potentially converting the entire biomass of the earth to such machines.
But let’s give Kurzweil the benefit of the doubt, and say that doesn’t happen—and that his cybernetic singularity actually does. Then what? Perhaps the most pressing problem is that it will be impossible to distinguish who we really are from our replicas or nonbiological extensions. In other words, what will ‘human’ even ultimately mean in the context of human control when we introduce a nonbiological addition to our own decision-making starting in the 2030s using brain-computer interfaces? That nonbiological component will only grow exponentially, while our biological intelligence will stay the same.
So, as we get to the late 2030s, our thinking itself will be largely nonbiological. So where will the human decision-making be when our own thought largely uses nonbiological systems?”
Will there be any way to tell the difference?
Shawn Gorman is Deputy Editor of News for Popular Mechanics. Before joining Pop Mech, he specialized in academic reading for educational testing. His diverse academic background also includes time spent as a lecturer and librarian at institutions such as the University of Kentucky, SUNY, and Harvard University. Shawn holds a Ph.D. in Modern Foreign Languages and Literatures from Boston University, an M.A. from The Ohio State University, and a B.A. in History from the University of Oregon. His favorite subjects to explore include computer science, physics, math, and biology. He lives on a medium-sized cat ranch in Easton, Pennsylvania, with his wife and two daughters.
