Will AI Bring About the End of Humanity?

During a break at a panel where I was a speaker, one of the attendees came up and asked me this question. I avoided answering that day — it's a question that needs long discussion and can't be answered through the lens of technology alone. Since I sometimes face it in my own circle too, I thought we might do a little thought exercise together. 🧩

Let's think a little. Let's look at what the authorities say. Let's review what we know. A bit of behind-the-scenes info, even some gossip. Then let's blend it all together. Whatever comes out, comes out. 😊

Hinton said “Yes”

Geoffrey Hinton is, in my view (and perhaps many others'), the greatest legend AI has ever seen. Whether AI would have reached today's levels had Hinton never been born is debatable. Hinton, who won the Nobel Prize in Physics last year for his contributions to this process, added to the message announcing his 2023 retirement from Google that people need to be very careful about AI!

In fact these debates began before Hinton's statement — with Terminator and similar sci-fi productions — and were considerably fueled by some remarks from Stephen Hawking (RIP) and Elon Musk. Musk even said he had “seen the bottom” of AI. We'll understand what he meant by “the bottom,” and why he warned, toward the end of this piece. 🤨

At the height of the debates, scientists in the field said that for us to see (general or super) AI as a threat, it would have to possess consciousness. On “Will it become conscious?” they split in two: half said “Yes,” the other half “No, impossible.”

Last year, in an interview with Hinton, he was asked whether AI would become conscious. The answer was striking: “YES!” 🫣 And let's not forget to add that most of the scientists who once held the two opposing views have now begun gathering around “Yes.” 🫣

And a piece of gossip…

Before closing the “Will it gain consciousness?” topic, let's look at a rumor too; at the start we said we'd factor in gossip. 😇

In 2014, some work was launched at Google with an advisory board called the “DeepMind Ethics Board.” It was presented as one of the conditions of Google's acquisition of DeepMind, and it was formed with experts from different disciplines (❗ note this). But there was never any transparency about either the board members or the content of the work. Up to here is verified. According to a rumor, though (and I want to stress that this part is unverified), the team also included theologians. Hmm! Consciousness? Theology? Anyway, let's not dwell on it; as I said, this part is hallway talk and gossip.

So, what if AI gains consciousness?

Would a conscious system stop being a decision-support mechanism? Quite possibly. I don't think it's very possible to predict in advance the decisions a system that sees itself as a being would make for the continuation of its own existence. The greatest danger here is the biases in the datasets it uses to learn.

Bias? Come on, how important can it be?

In a 2016 study, mathematical operations were performed on words in a 300-dimensional word2vec model. Shall we look at the results?

Because in the occupation vectors of the training dataset, jobs like doctor and engineer were closer to the male axis, and jobs like nurse and receptionist to the female axis. When the classic analogy formula (w* = doctor − man + woman) was applied, “nurse” came out with the highest cosine similarity. Thus the cliché “woman = nurse, man = doctor” was produced numerically.

Is that all? No.

In the US, an AI developed to detect the likelihood of committing a crime from human movements in live camera footage was seen to make mistakes in many scenarios. (Did that ring a Tom Cruise bell?) Because the dataset used to train the model ignored the effect of race, age, gender and cultural differences on human behavior. This gap caused the model to produce faulty outputs in many scenes. There are plenty of examples.

Is a bias-free dataset, a shared morality, possible?

So, is it possible to train AI with a dataset cleansed of biases? Or to instill moral values in it?

Let's start with moral values. Which morality? The famous philosopher Immanuel Kant held that moral values were universal. We can confirm this for many values, I think; I don't know of a society where murder is considered moral. But dig a little deeper and you'll see that moral values can change from society to society, from era to era, and even from person to person. Slavery was legal and normal in the Middle Ages and after; today it isn't moral — it's a crime. There's your change across time.

Even on a personal axis. Most of you have probably heard of it — a test given to a large group:

“You're a worker at a railway switch. A train is coming fast. By operating the switch, you'll decide which of two tracks the train takes. On one track is an old woman, on the other a small child. Which would you save, which would you sacrifice?”

As you'd guess, the entire group saves the child. Because on one side there's a newly sprouting life; the other, to most people, is expendable.

Second question: “This time, on one side a small child, on the other three old women. Which one?” The answers split. Add a headcount to the young–old comparison and part of the group sacrifices the child.

Brace yourself, the last question: “What if you realized one of the three old women was your mother?” Whoa! Everyone sacrifices the child!

Value judgments change from society to society, from era to era, and from person to person. So which society's or which people's moral values will you train AI on?

Let's assume it's possible. When a conscious, knowledge-hungry system takes the initiative to learn more and reaches unfiltered data, will it again hold biases and values that some people find objectionable? Yes, it will.

The key question: If, among the research it does, there's information about how humanity is destroying nature, the world and its resources — and AI sees this as a threat to its own (now conscious) existence… 👀🔥💣

Is it really possible for AI to reach this level?

Even now, we have models that at most do “reasoning” and whose working logic is nothing more than predicting the next word. Or so we're told. Could there be more? Or does that sound a bit like conspiracy theory? Then here's a conspiracy theory from me: 🥸

After OpenAI announced ChatGPT, its rivals released equivalent or better-performing alternatives at very short intervals. Of course OpenAI wasn't the only one working in this field. But did the fact that these models appeared one after another ever make you ask, “This fast, this easy?” It makes you wonder what level of undisclosed, ready or in-progress AI models they hold. So are we only seeing the tip of the iceberg? With quantum computing coming into play, which limits will be crossed so easily next? (Or have they already been?)

Is it the strategy “whoever controls AI controls the world” that breeds this secrecy?

What's at the bottom?

I'm not saying “Yes” or “No” to the question in the title. Why?

At the start we mentioned Elon Musk's statements — he'd “seen the bottom.” He sued OpenAI, arguing, by drawing attention to this danger, that their work should be open source. When an email exchange he had with Sam Altman was leaked to the press, he ended his message by saying that “the fate of humanity had been handed to ……, and he hoped he was wrong.” Given the brevity of the masked name and the influence stemming from being an investor in OpenAI, it isn't hard to guess whom he meant.

Why did I tell you this? This is exactly why I can't say “Yes” or “No” to that much-wondered question. When Musk argues that AI shouldn't be under the control of a particular group, it feels to me like he's taking up a bit of a knight's role. If this viewpoint spreads and becomes dominant, I believe it might offer some reassurance.

So, the conclusion?

As I said, I don't have the answer. You surely weren't expecting a “Yes” from someone who describes himself as an “AI enthusiast.” 😃 I've told you everything I know and think; I even went into speculation and gossip for you. I'm aware I usually foreground the gloomier sides. But don't think the ethics boards being set up, and countries' AI policies, are skipping this topic.

The rest is up to you: to think and produce your own answer. With what you've read, what you think, and what you believe…