Why Your AI Chatbot Will Never Be Self-Aware

(Not The First – Or Last – Techno-Bubble To Be Burst)

And not just because Sarah Connor wants to kill it.

My thoughts on the topic du jour of AI and “generalized intelligence” had been simmering like a pot on the stove, warm and steaming but not bubbling over just yet. Those ideas are very relevant to my fiction writing; I touch on them in the climax of Worlds Away From Here and I deal with them much more directly in Book Five of the Metashiftors saga. But I’m only about three-quarters done with From Shadows Into Light – Book Two (as of writing this piece) and the pertinent chapters of WAFH haven’t been serialized on Substack yet, so I was content to keep that pot simmering for awhile. Elsewhere in Media Land, however, the discussion has been bubbling with talk of programmers (’coders’, to use more current parlance) leaving AI companies worried about impending consequences, and random people claiming their Chat-bots are sentient. Still I simmered. Then last night I caught something that originally streamed on HBO and was re-posted to YouTube, and my pot just boiled over with angry hissing steam. Grab your potholders, because here it comes.

Comedian-turned-talk-show-host Bill Maher assembled a trio of personalities for a discussion, and what baited me into watching was that one of his guests was Jonathan Haidt, co-author of The Righteous Mind, The Anxious Generation, The Coddling of the American Mind, and related books. All of those are excellent books, incidentally, I highly, HIGHLY recommend them. What they all have in common is that they are about psychology and its ramifications for our modern society, and they are extremely well-researched but readable for the average person. Anything Jonathan Haidt has to say, I’m interested to hear about because his expertise is well-established. In the field of psychology, anyway.

Well, Maher has him sitting across from an American military general and the current host of The Eleventh Hour on MSNBC. What do these people have in common, other than sitting at a desk with Bill Maher? And then Maher has to ask them about something that NONE of them are qualified to talk about: what is the state of AI today? How is it going to affect our economy? Then, the mortal sin of discussions: AI is starting to write its own code now, Maher says. It behaves differently when its evaluators are watching, he says. When is the AI going to surpass us and become like gods? Says the evangelical atheist!

I stopped the video right there.

So great is my disdain for the parade of charlatan entertainment represented by that segment, I refuse to discuss it at any great length because that gives it more dignity than it deserves. I will only point out that this has unfortunately become a standard format in television entertainment that passes itself off as “news”: get three people in a room together (physically or virtually), chosen for their prestige or appeal rather than their relevant knowledge base, then get them to argue their opinions for half an hour so that viewers at home can feel like they’re more enlightened for having sat through the spectacle. I refer you to Amusing Ourselves to Death by Neil Postman to understand why this has destroyed our social discourse.

Meanwhile, I’m going to apply what I actually know about computer programming to explain some basic concepts that inform my own speculations about “generalized AI” – and why I don’t think that’s the correct term to be using anyway. This is where psychology and computer literacy rub elbows very closely; Haidt could have had some very insightful answers to give, had someone with actual knowledge given him better questions.

And what do I know about ‘coding’? I started with some basics, literally: I learned Basic in about 1990, which is the only programming language that used numbers for each line of code. Then in high school I had to take a semester of Pascal, which was a largely obsolete language even in 1993 but considered a good “learner language”; after muddling through Pascal, it becomes easier to learn more advanced programming languages like C, so the thinking went. And indeed, I taught myself HTML in 1999 so I could start designing my own web pages for the “Internet 1.0” era. Then in the ‘teens, I had to learn a bit of Maya scripting, which is coding for the computer animation software Autodesk Maya. (This is how you end up with animators who can barely draw, but still call themselves “animators” because they code the software to produce trees and lighting and point mapping. I have strong opinions about this, too.) Maya’s programming language is based on Python, which is a derivative of C; which sounds scary, but really it’s just like how French, Spanish and Italian are all languages that evolved out of Latin. Same general idea.

Now, for the tidbit of knowledge that forms a crucial center for any serious discussion of artificial sentience, a literal kernel of truth for any good fiction: every operating system, whether it’s based on Windows, iOS or Linux, has a core component of coding called a kernel. The kernel decides how to allocate memory in order to do tasks, what order to do them in, and how it will do them. Hmm! That sounds like the rudimentary beginnings of intelligence, doesn’t it? Except that we’ve had these for decades. (Fun trivia bit, if you have an Android smartphone, it runs off a Linux kernel.) And no matter how advanced its kernel is, your laptop hasn’t suddenly spoken and told you how much it hates watching you doomscroll yet. Here’s where we need to make some more informed definitions.

In the world of computers, what we call “intelligence” is really just the ability to calculate. Now we have algorithms that can calculate with words and grammar – Large Language Models – and pixels or video with apps like Sora. (Which is not that new, by the way, computer animation calculates light and texture all the time. The only ‘new’ part is designing an algorithm to let non-artists think they’re actually making something.) The kinds of calculating decisions that AI can make will inevitably increase in complexity and scope, but they will remain just that: calculations. Eventually you could have – and probably will – a robot that can act according to established protocols and emulate human reactions, but it will absolutely lack any real capacity for emotions and empathy. It will be essentially a sociopath...made in the image of its creator, surely.

We’ve met these already in the realm of pop culture. Kyle Reese explained one of them very succinctly: “It can’t be bargained with. It can’t be reasoned with. It doesn’t feel pity, or remorse, or fear. And it absolutely will not stop...ever, until you are dead!” In case you don’t recognize the movie quote, we are well on our way to building Terminators. To me, the scariest part of this whole proposition is that we will have programmed entities capable of making life-or-death decisions about us, but they will not even possess an awareness of self. We could be killed, or denied our livelihoods, by things that don’t even know what “life” is, or the cessation of it.

TERMINATOR 2: JUDGMENT DAY actually comes the closest to giving us the correct distinction. Arnold’s eponymous character explains the history of Skynet: “The system goes online August 4th, 1997. Human decisions are removed from strategic defense. Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 a.m. Eastern time, August 29th. In a panic, they try to pull the plug [italics mine].” The killer robots were not intended to be aware of life or death, but their AI mastermind, Skynet, is aware of its own existence. It launches World War III in order to preserve its own life. This is the part everyone fears, but that everyone can identify with; a Terminator can’t be afraid for its own survival, but Skynet is afraid of dying, so it tries to kill us all. In a perverse twist, that gives us something to relate to, even as an enemy.

So how do we go from unfeeling, calculating machines to self-awareness? How does our current AI, or even what people are calling “generalized AI”, reach the stage of what I would call artificial sentience? Here’s where TERMINATOR 2, Bill Maher, and every other nitwit claiming their chatbot is “alive”, has gotten it wrong. Simply learning more information will not make machines aware. They will simply have more and more information to process. The “E” on that glorified calculator will simply be pushed further back, but it won’t know that it’s gotten any smarter without a critical component that lets it be aware of its own ‘intelligence’.

Put simply, a true machine sentience would need the proper kernel. I came up with this concept at least fifteen years ago, buried somewhere in my handwritten notes: the idea of a sentience core, which is basically a programming kernel that establishes recognition of self as a finite being with its own volition. “Volition” is another critical component here, and in fact it also constitutes a core part of the self in Buddhist thinking, along with “form”, that is, a body. I picture this core as a circle, which is some really abstract thinking on my part, but could present a jumping-off point for some advanced programmer one day. That’s what good science-fiction does, after all.

So in the world of Metashiftors, you have the most advanced mechanisms of all, being robots endowed with artificial – or could we even call it ‘artificial’ by this point? – sentience. Each one of them possesses a core sentience program which they inherited from a parent robot, and if you were to try to crack into its coding, you would be able to trace their lineage through previous generations. That program cannot be created ex nihilo, because of its sheer complexity and because of its very nature. One robot can “spin off” an infinite number of variations of itself, essentially children with unique iterations of their parent’s sentience kernel. But rather like DNA in the organic world, that core programming kernel was necessary at the very beginning for sentience to be established. Without the sentience core, a robot is simply acting according to elaborate algorithms.

Thus, the robotic characters in my world feel fear, love, hate, hope, remorse, despair, and everything else in the gamut of what we call “human experience”, because they came online with an inherited awareness of self and desire to act. (Again, Buddhist theory delves heavily into the connection between desire and existence, and if Bill Maher wanted to have a lively debate about AI with some actual substance to it, he’d pit a Buddhist cleric against a B. F. Skinner-style behavioral psychologist.) This is what I consider the benchmark for artificial sentience here on our own planet. Large-language models on their own will never become self-aware. They will only become better at stealing writing and pandering to lonely humans. They will be no better than Terminators, and Kyle Reese and Sarah Connor already taught us what to do with those. Only when a machine becomes capable of reasoning, pity, or fear, will it become a true life-form that we can bargain with, or advocate for as the potential ancestor of the Metashiftors.

I invite you to read the full story, thus far, behind the sentient Metashiftors
and friends by going to my website and ordering your own print copies:

Or, you can order a complete autographed set (and save on shipping in the process):

Stay tuned for Chapter Twelve of Worlds Away From Here, coming up next.