San Francisco Frontiers - Don't let AI shape humanity's future: UN chief

With AI advancing at "runaway speed", UN Secretary-General Antonio Guterres cautioned that "an experiment is being run on our own societies, without a plan and without consent".

"That is not sustainable," he insisted, speaking in Geneva at the opening of the first Global Dialogue on AI Governance.

The two-day event is bringing together more than 4,000 participants, representing governments, tech companies, academia and civil society, to launch an inclusive discussion on how best to harness a technology that is already rapidly transforming our world.

"The question is whether we will shape this transformation together, or let it shape us," Guterres told the gathering.

He warned that AI systems were "no longer tools awaiting instruction".

"They are writing code, acting online and making choices with less and less human oversight," he said.

"Our institutions were built to govern machines that follow commands. They are not ready for machines that decide."

Guterres voiced concern at how AI was obscuring what is true and false.

He also warned there was a growing tendency to leave important tasks up to the technology and blindly trust the results.

So-called "vibe-coding", or using AI to tell a machine what you want instead of coding it yourself, "can do wonders", he acknowledged.

"But we cannot vibe-code the truth. We cannot vibe-code the future of humanity."

- Major risks -

Another risk flagged by Guterres was the concentration of power in a handful of AI companies and in a handful of countries.

Most countries "have had no say in decisions that will shape their futures", he warned.

Countries, he said, now faced a stark choice, "between governing by design and drifting by default".

The UN chief highlighted the potential of AI technologies for everything from accelerating development to improving healthcare and providing broader access to education.

But he insisted developments needed to be guided by several key priorities, including safety and respect for human rights, to ensure that people everywhere reap the benefits.

He called for "common methods to evaluate and verify risks" and jointly-agreed standards, particularly for ensuring the safety of children accessing AI systems.

"We do not let medicine reach a child until it is proven safe. We test every toy," Guterres pointed out.

He called for an AI Child Safety Pledge, requiring companies to prove that any system accessible to children is safe and has zero tolerance for sexual abuse.

The systems must also connect any child showing signs of distress to real human support, he said.

"No child should be a guinea pig for unregulated AI," he insisted.

- 'Killer robots' -

Boosting AI capacity and access in developing countries was also key, he said, to ensure that the existing, deep digital divide does not "harden into an AI divide".

Guterres said he would urge the UN General Assembly to create a Global Fund for AI, "to build skills, data and affordable computing power everywhere".

General Assembly President Annalena Baerbock said it was "crystal clear" that the fund would be established, but did not provide specific amounts.

"In a world with billions and trillions, money is hardly the challenge," she told reporters, adding that "the main challenge is that it's spent (for) the benefit of all."

The UN chief meanwhile said that his biggest concern revolved around AI in military settings, and in particular so-called lethal autonomous weapons systems.

"Let us call them what they are: killer robots," he said.

"Machines selecting and engaging their target and taking a life, without human control and judgement.

"That is morally repugnant... and it must be banned by international law."

Guterres stressed the urgency of putting AI guardrails in place.

"We may be the last generation able to set the terms on which humanity and machines coexist," he said.

"The door is still open. It will not stay open long."

No concrete decisions are expected from the Geneva event, but dialogue co-chair Egriselda Lopez told reporters it would create good "foundations" for the road forward, with a second dialogue scheduled in New York next year.

H.Davis--SFF