I could only stand Sam Harris for ~15 minutes. The paranoia was too vague to assess.
Computerphile was much more interesting and had what I'd consider the most telling statement.
Early in his video:
"If we have a general intelligence, we have to make sure that its preferences are aligned with ours."
This is what I'd say is the crux of the biscuit- motivation, goal direction. Yes, a general intelligence like the one postulated could kill us all by accident.
If one existed. There is some sort of assumption here that as soon as enough computing power is available, it will immediately, recursively optimize itself into godlike power and break away from human control and understanding. There never is any explanation of how or why it could do this.
See Lawnmower Man or Colossus, the Forbin Project.
I'm much more worried about some central banker or DARPA will give direction to a very powerful but not godlike computing device to use it as a tool to economic or military world domination. I doubt we'll ever get that far. Ecological with resulting societal collapse from miscalculations made during the industrial age is more likely to land us back in the neolithic or early bronze age.
Computerphile was much more interesting and had what I'd consider the most telling statement.
Early in his video:
"If we have a general intelligence, we have to make sure that its preferences are aligned with ours."
This is what I'd say is the crux of the biscuit- motivation, goal direction. Yes, a general intelligence like the one postulated could kill us all by accident.
If one existed. There is some sort of assumption here that as soon as enough computing power is available, it will immediately, recursively optimize itself into godlike power and break away from human control and understanding. There never is any explanation of how or why it could do this.
See Lawnmower Man or Colossus, the Forbin Project.
I'm much more worried about some central banker or DARPA will give direction to a very powerful but not godlike computing device to use it as a tool to economic or military world domination. I doubt we'll ever get that far. Ecological with resulting societal collapse from miscalculations made during the industrial age is more likely to land us back in the neolithic or early bronze age.
So how, exactly, does God know that She's NOT a brain in a vat? 
