April 1st, 2019 by

Jaan Tallinn is a founding engineer of Skype and Kazaa. He founded the Centre for the Study of Existential Risk and he is also a co-founder of the Future of Life Institute and philanthropically supports other existential risk research organizations.

He was recently interviewed by The Guardian on the topic of  AI’s potential to end human civilization. This is an excellent – albeit long – read, with some juicy quotes and insights. Here are some of my favourite snippets below:

“I do agree that … preparing for the event of general AI surpassing human intelligence is one of the top tasks for humanity.”

“If the reward structure of a superhuman AI is not properly programmed, even benign objectives could have insidious ends.”

“Superintelligent AI brings unique threats. Ultimately, I hope that the AI community might follow the lead of the anti-nuclear movement in the 1940s. In the wake of the bombings of Hiroshima and Nagasaki, scientists banded together to try to limit further nuclear testing. “The Manhattan Project scientists could have said: ‘Look, we are doing innovation here, and innovation is always good, so let’s just plunge ahead… But they were more responsible than that.”

“Tallinn proposed not only walling off superintelligence in a holding tank – a physical structure – but also restricting it to answering questions, like a really smart Ouija board. The approach that most excites researchers is finding a way to make AI adhere to human values– not by programming them in, but by teaching AIs to learn them. In a world dominated by partisan politics, people often dwell on the ways in which our principles differ.”

When asked what success in AI safety may look like, Tallinn made reference to the Lego movie where “…everything is awesome.”

Guest post by The Futures Agency content curator Petervan

Other Resources

Author: Gerd Leonhard

In the words of American poet John Berryman, “the possibility that has been overlooked is the future”. Most of us are far too busy coping with present challenges to explore the future in any depth – and when we do our own cravings and fears often run away with us, resulting in utopias or dystopias that are not very helpful in terms of planning and decisions. Today’s professionals, leaders and their organisations need a dedicated, passionate long-term understanding of the future if they are to successfully navigate the exponential waves of change. For countless individuals and organizations that intelligence is called Gerd Leonhard.

Share it