Wiki Interview With Eliezer/Issues Of Concern
From The Transhumanist Wiki
Issues of Concern
Are you concerned with the possibility of strict regulation on AI? If so, why?
Perhaps I am overly pessimistic, but I see absolutely no hope that a government committee could legislate Friendly AI. Friendly AI is a basic research issue of equal depth with General Intelligence and seed AI, not as large, but just as deep. If a government committee cannot build an AI, they cannot figure out how to make it Friendly. As for the "relinquishment" of AI, Bill Joy's euphemism for bans, as far as I can see this will either transfer the development of AI to rogue states and organizations, or else delay the development of AI until one of the other Existential Risks wipes out humanity. I am concerned with the possibility of the regulation of AI because the expected outcome of such regulation is catastrophe.
(copied to Government Regulation Of AI)
Does the possibility of a call for relinquishment on AI development, as has been done with cloning, affect your present-day decisions and actions?
Because there are many people acting in the realms of politics, and even a substantial number of transhumanists doing so, my action in this realm tends to be stating the argument against government control of AI design features, and occasionally minor things like posting the first comment in the Reason debate between Stock and Fukuyama.
(copied to Government Regulation Of AI)
If you believe regulation of AI in the future represents catastrophe for us, then what do you propose is done to prevent said regulation?
I believe that if the creation of AI is important, then given my skills and the existing balance of investment, most of my efforts should go into accelerating AI, rather than political opposition to the regulation of AI. It seems unlikely to me that AI will be banned in the next five years, but it is possible - the question for me is not "likely" or "unlikely" but what I can do about it. I suppose that I would recommend that others carry on the fight against regulation of genomic technologies, and nanotechnologies if applicable. AI currently does not appear to be on the radar screen, but I suppose I would recommend fighting against current efforts to mandate features in all computer hardware and software. I think the initials of this particular evil plot were SSSCA (Security Systems Standards and Certification Act). In the end, the government is too large to push on and too blind to persuade, so while I do think that opposing malregulation is important, I would not count on it working.
(copied to Government Regulation Of AI)
Do you believe it will be a more negative or positive net-effect on SIAI's efforts when millions of people have knowledge of the singularity and related concepts?
The potential positive effect is increased support and funding. The potential negative effects include malregulation and "pollution" of the meme by attempts to repackage it for what the salesperson perceives as the lowest common denominator.
Do any individuals or groups presently represent implicit or explicit opposition towards the achievement of the Singularity? If so, please elaborate.
Leon Kass and Bill Joy are both certainly implicit opposition. Any group that dislikes science/technology (and yes, they are antiscience and antitechnology, despite some attempts to position themselves otherwise for PR purposes - Greg Burch has an excellent analysis of the ongoing fallout from Rousseau) Anyway, any group that dislikes science and technology also tends to dislike rational thinking, meaning that they are unlikely to be able to see the Singularity as anything but yet more science and technology, and that they are likely to blindly oppose it without being willing or able to grasp the consequences of their actions. So far I think that most of them would be unwilling to accept the concept of a Singularity to begin with, regarding the acceptance of the idea as proof in itself of technolatry, and I'm fine with that, but again, these are all such massive events that it is much easier to influence the future by developing the technology than by playing politics. As seen, I sometimes do try to intervene in politics myself, but only targets of opportunity, like the Reason debate.
(copied to Opposition To SIAI)
Specifically, why do Leon Kass and Bill Joy represent implicit opposition to the achievement of the Singularity?
Anyone ragging on the evils of the "posthuman" is an implicit opposition to the achievement of the Singularity. Basically, I would say that implicit opposition to the Singularity is anyone who believes that the future ought to be just like the past, and who believes that suppressing technologies is a safe and effective strategy for doing so.
(copied to Opposition To SIAI)
In PtS, you wrote, "The best candidate for creating the Singularity is Artificial Intelligence; the technology most likely to wipe out the human race is nanotechnology." What is your reasoning behind the claim that nanotechnology is the most likely candidate to "wipe out the human race"? If possible, please provide references to support your claim.
I suppose Nick Bostrom's "Existential Risks" paper might count as a reference. "Misuse of military nanotechnology" is ranked first there. Frankly I fail to see why quoting this should prove anything, but never mind. Anyway, things have changed since PtS. Nowadays biological warfare, which I think was formerly Existential Risk #3, seems to be moving up on everyone's agenda. A full analysis of "What happens when nanotechnology gets developed?" is just too complex to do in this interview. PtS will have to stand on its own there, but I'm not sure anymore that nanowar should be Existential Risk #1. The computing capacity offered by nanotechnology may mean that AI is necessarily developed in advance of nanowar becoming possible. In that case you'd have the separate problem of whether AI can be brute-forced but Friendly AI can't be brute-forced, and whether AI can be brute-forced in advance of medical nanotechnology providing for brain-computer interfaces.
In response to Bill Joy's essay, you expressed the belief that military nanotechnology will have more offensive or destructive capabilities than defensive capabilities. Why? John Smart claims that there is growing systems theory evidence that the most complex local substrate or system, at any given time, will have decreased its information loss or destructive capability more so than any other previous substrate. Do you believe this is a valid argument in relation to the proposed real-world destructive capabilities that different advanced technologies will have, as well as their increasing accessibility to individuals or small groups?
To be honest, this sounds to me like complete wishful thinking. It sounds to me like a prediction that World War II was less destructive than World War I, that World War I was less destructive than medieval wars, and so on. As technology improves, our creative capacities and our destructive capacities both increase. This has to be balanced with improvement in intelligence. Obviously it would be very nice to conclude that defense will outweigh offense, but it is totally untrue in modern times. (i.e., no defense against nuclear weapons) It is not supported by Freitas's paper on grey goo, in which enormously more sophisticated defenses are needed to defend against even accidental outbreaks of nanotechnology. You'd need one hell of an argument from complexity theory to outweigh those facts, and I would expect such an argument to be expressible constructively, i.e., for it to explain concretely how an active shield can fend off military replicators with the capability to construct fusion devices. I think John Smart relies far too much on analogies from complexity theory, but then I've said this already.
(copied to Wiki Commentary On Why The Future Does Not Need Us)
In response to Bill Joy's essay, you wrote, "Either way, I think that greater-than-human intelligence is the only force powerful enough to match the massive social, economic, and technological forces pushing humanity towards a replay of World War II fought with nuclear and nanotechnological weapons. The question is whether a greater-than-human intelligence would be kindly disposed towards humanity." What social, economic, and technological forces do you believe are guiding us towards global catastrophe? Why are they guiding us towards catastrophe? Why do you believe we exist in a meta-stable state?
I am slightly more optimistic today. It could be that if we lived in Vernor Vinge's "Slow Zone" where all transhuman intelligence is magically prohibited by the laws of physics, that humanity would be able to spread off Earth and survive indefinitely. This is ruled out as a possible future because we do *not* live in the Slow Zone - I emphasize that lest anyone become emotionally attached to it. I would still expect a major catastrophe to wipe out most of Earth at some point or another, because planets are just too vulnerable to accident and attack, but humanity would survive. When I wrote the original response to Bill Joy, I think that I was considering the destabilizing effects of the incremental development of nanotechnology, but not considering possible stabilizing effects. That is, I then expected the final development of true machine-phase replicator technology to come at the tail end of a long period of rising international tension driven by the military deployment of incremental technologies, rather than considering the alternate possibility of replicator technology developing at the tail end of a global economic boom driven by the civilian deployment of incremental nanotechnologies. Today I would have to say that it could go either way. We could get WWIII even before the development of nanotechnology, just as the result of destabilizing development of incremental technologies and fear by various nations that some other nation will develop replicating nanotechnology first; or the Singularity could happen a couple of weeks after SIAI gets its mitts on a nanocomputer; or nanocomputers could be militarily restricted technology and some DARPA project could brute-force an unFriendly AI; or nanocomputers could be a militarily restricted technology; people might be smart enough to avoid Brute-Forcing AI without a theory of Friendliness; and human-computer interfaces could be developed with medical nanotechnology - after which the augmented humans would steer the Singularity from there; or neither form of intelligence enhancement could materialize, and economic prosperity and the devastation of whole countries by military nanotech could go in tandem; and so on, but there is still the underlying dynamic of a metastable civilization in which the two attractors are superintelligence and extinction, as discussed before. In the end you can see all technological changes by how they advance either or both possibilities. Destruction gets cheaper along with construction. So far technology has always been a net benefit to humanity because more people with access to technology are constructive rather than destructive. This is a product of a social structure that tends to punish destruction, and of our ability to play constructive (positive-sum) social games where beneficial, and even of those emotional drives toward altruism which are culturally reinforced by prevailing memetic contexts. But there are also contexts in which humans play negative-sum games, such as war. If nuclear weapons become cheap enough that small, rogue nations can afford them, then I think it will not be too long before they are used. There are worse problems than that. There are cases where it seems as if a small group or an individual may have the potential to wipe out the world. Biological warfare is the most obvious example. It is possible that human society is smart enough to dance around this forever, but I would not wish to rely on it. We must eventually confront the challenge of transhuman intelligence in any case. This leads me to see improvement of intelligence as the escape hatch from the existential-risks game, and to believe that we should confront it sooner rather than later.
(copied to Wiki Commentary On Why The Future Does Not Need Us)
We haven't explored in detail the reasoning behind your claim that our metastable species has only two stable attractors, superintelligence and extinction. Please elaborate on this claim.
Can you name another stable state? Superintelligence is an attractor because increased intelligence begets increased intelligence. Extinction is an attractor because an extinct species stays extinct. That's two. I don't see any others offhand.
