Tuesday, February 28, 2006

Outwitting Rogue Superintelligence

"Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an 'intelligence explosion,' and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make."
-I. J. Good, 1965

Superintelligent machines are expected to usher in an age in which the pace technological innovation outstrips our ability to comprehend it. This era is known as the Singularity.

You say this Singularity thing sounds dangerous? Give that man a cigar Mars bar!

The key issue is whether runaway intelligence and technological prowess is something against which we can defend ourselves.

The worst strategy is to try to ban AI technology. To do this would require a totalitarian regime that would be worse than any rogue super AI.

Another strategy is to race to be the first to build a superintelligence so that you can ensure its friendliness. This is tricky, but not impossible. This is the goal of the Singularity Institute for Artificial Intelligence.

Today, it occurred to me that there might be another strategy. There might be a way to tie together many independent computers into a giant distributed supercomputer. The idea being that no single computer, or subcluster could oppose the combined defensive computing might of this distributed system.

There are two dangers to this approach. The first is that, in acting to defend itself, the defensive network might supplant the threat. The hunted becoming the hunter, so to speak.

The second danger is that such a network might be pre-emptively hacked and controlled by a superintelligence.

So, it's not really a perfect plan, but I wonder whether there might be some workable variation on this "defensive swarm" theme.


Peg said...

Hi Doc, couldn't help but chime in about a book written by Dean Koontz many years past about a scientist who wanted to combine intelligence with machine...there is a book called the Demon Seed by him, but I don't think that is it...this book is old. But anyway it was interesting to discover what the "entity" really was.
Just reminded me of that! Spooky crap nonetheless and Dean Koontz is from Erie, Pa. Well goodey goodey gumdrops, right? ;-]

Doctor Logic said...


I just looked this up and found that they made a movie version of the book in 1977 (starring Julie Christie).

Are you from Erie, PA?

Peg said...

No, that wasn't the book. This book started out in a town where the rich guy owned it and set up a computer in every household. His ultimate goal was to 'meld' every computer and person to become one one ultimate machine (brain) and took everyone back to their original origin and so forth and so on....they all become amebas in the final stages of their life...you have to read it, I don't do it justice but I still can't remember the name of it!
Anyway, no, I am not from Erie, about two hours away as a matter of fact, but PA nonetheless! I just find it interesting because the hubbub is that people from Erie are eerie and wooolah, Dean Koontz is a resident! But, I don't believe it myself... I find they are everywhere!!!!! ;-0}