“We don’t know how to train AI to ensure human values”

“ Simplicitly we do not have the knowledge and technology to understand how to train AI systems [artificial intelligence] to ensure that they have the types of goals, values or motivations that support human flowering,” says Malo Bourgon, who spoke to Lusa from Berkeley, U.S.A., in explaining his concern about the accelerated development of technology.

Moreover, “we don’t even know if we know what we want the AI to care about, how to successfully train them in a way that we can be sure that these things are really what the AI has learned to value.”.

And even if we knew that, “we still have big open questions about what these things would be, if, by creating something so powerful, it would end well for us.”.

Therefore, “there are two levels of problem and we have not made much progress on any of them. And that is what I would like people to imagine: imagine, basically, aliens to appear and, if they have very advanced technology and do not care what we care about, we would be at the mercy of the way they wanted to transform the planet” and “there is not much we can do about it,” he points out.

Malo Bourgon stresses that “being intelligent is something very powerful,” recalling that humans compared to other species on the planet are more intelligent.

Although humans are no longer, nor intend to harm other species, it is certain that throughout the history of mankind they have caused the extinction of “more than 10,000 species” due to the goals and ways of reshaping the world for its purposes, he recalls.

Now, “being very intelligent and very efficient in pursuing goals will allow AI systems to reshape the world in powerful ways by pursuing their goals. In addition, the way we train these systems today — the technology and methods we use — does not allow us to understand what they have learned or what goals, values or impulses have emerged from this training process,” argues the institute’s CEO.

When systems are tested and evaluated in various ways, they seem to “resist” the shutdown and change of objectives and “have instincts to acquire more resources”.

Since “they are at their current level of capacity, this is not so worrying. But if they are radically smarter than us — and I believe that if this continues, we will not be talking about AI systems as intelligent as Albert Einstein is about me, but about systems as intelligent as me (or you) in relation to a chimpanzee, a rat or an ant — then if such an entity (or several of them) is operating in the world, it will be extremely difficult to control something much, much smarter than you,” he explains.

And unless this entity “worries about things compatible with human flowering”, its actions in the world will be “radically destabilizing and likely lead to changes in the functioning of the world that are incompatible with our continuity,” the MIRI CEO, who has existed for about 26 years, considers.

“I would say that we were one of the first organizations to really consider the idea that, at some point, humanity could create AI systems with general capabilities and intelligence — in a way similar to the intelligence and general capabilities of human beings” and “during most of our journey, we helped consolidate the security area in AI” and its alignment.

“ We helped coin this term and boost this area of study,” he said.By 2020/2021, they began to realize that the advance of AI was very fast, but progress in this area did not follow the pace.

Meanwhile, when ChatGPT was launched “we redirected our efforts to communication and governance.”.

Basically, “we tried to use the experience we have accumulated over more than 20 years to reflect on these challenges and problems to help people understand the challenge we face and the risks — risks that some of the great minds in the area, in the 1950s and 1960s (when the AI field was just beginning), like Alan Turing and I.J. Good, already predicted by imagining what it would mean to achieve the goal of building machines capable of thinking,” he says.

The idea of “recursive self-improvement — with AI improving themselves — seemed like a real possibility. Thus, even before most of the science fiction that people today quote by suggesting that this is all just fiction, the experts who reflected on what it would mean to succeed in the area of AI already considered these issues as natural consequences with which we would have to deal,” he said.

“Science fiction is sometimes very good at understanding what the trajectory of future technologies can be,” so “not all science fiction remains only fiction forever,” he adds.