How would we build a really complex system — such as a general artificial intelligence (AI) that exceeded human intelligence?
That is the question Steve Jurvetson addresses in yesterday’s Technology Review. He considers it as a choice between two options: evolutionary search algorithms or design, and his summary of the problems with each are illuminating:
… designed systems also tend to break easily, and they have conquered only simple problems so far.
In fact, biological evolution provides the only "existence proof" that an algorithm can produce complexity transcending that of its antecedents.
But evolved systems have their disadvantages. For one, they suffer from "subsystem inscrutability." That is, when we direct the evolution of a system, we may know how the evolutionary process works, but we will not necessarily understand how the resulting system works internally.
If biological evolution provides the only proof that evolutionary algorithms can produce complexity transcending that of its antecedents, but biological evolution happened by virtue of evolutionary algorithms producing that complexity, are we in some slight danger of circular reasoning?
Another question: how much inscrutability does positing unknown/unknowable evolutionary processes for a system add– in particular, processes we only know of because we assume that that system was produced by evolution? Is design or evolution more likely to be a science stopper in going into further research?
The article (available here) is worth reading and thinking about.
Update: ID the Future has a post up on the problems with Jurvetson’s analysis.