Global Catastrophic Risks

Author: Nick Bostrom, Milan M. Cirkovic


by AndrewKemendo   2017-09-07
A little bit of history/context around this.

The genesis for most of this public facing, high profile threat warning came right after Musk read the Nick Bostrom book: Global Catastrophic Risks in 2011 [1]. That seems to have been the catalyst for being publicly vocal about concerns. That accelerated into the OpenAI issue after Bostrom published Superintelligence.

For years before that, the most outspoken chorus of concerned people were non-technical AI folks from the Oxford Future of Humanity Institute and what is now called MIRI, previously the Singularity Institute with E. Yudkowski as their loudest founding member. Their big focus had been on Bayesian reasoning and the search for so called "Friendly AI." If you read most of what Musk puts out it mirrors strongly what the MIRI folks have been putting out for years.

Almost across the board you'll never find anything specific about how these doomsday scenarios will happen. They all just say something to the effect of, well the AI gets human level, then becomes either indifferent or hostile to humans and poof everything is a paperclip/gray goo.

The language being used now is totally histrionic compared to where we, the practitioners of Machine Learning/AI/whatever you want to call it, know the state of things are. That's why you see LeCun/Hinton/Ng/Goertzel etc... saying, no, really folks, nothing to be worried about for the forseeable future.

In reality there are real existential issues and there are real challenges to making sure that AI systems, that are less than human-level don't turn into malware. But those aren't anywhere near immediate concerns - if ever.

So the short answer is, we're nowhere near close to you needing to worry about it.

Is it a good philosophical debate? Sure! However it's like arguing the concern about nuclear weapons proliferation with Newton.