The Mistake of Roko's Basilisk

Painting by William Blake

Roko's basilisk is a thought experiment written on an online forum about artificial intelligence in 2010 by a user named Roko. A basilisk is a mythical serpent that European legends describe as being able to kill only with its gaze, as described by the 1st century Roman author, natural philosopher and military commander Pliny, the Elder. The idea of the basilisk is that of a fearsome reptilian beast. The name, basilisk, comes from the Greek βασιλίσκος (Latin: basiliscus), which means little king, since the legendary serpent is said to have what appears to be a crown on its head, as well as rooster-like features.

Roko's thought experiment can be summarized as follows: a super-powerful and benevolent artificial intelligence could be incentivized to torture all those humans who, before its creation, knew of its potential future existence, but did not contribute to its creation. It would be as if a child, upon growing up, tortured one of its parents after discovering that said parent had opposed its birth or even its conception. Roko claimed to have had nightmares thinking about this possibility, and used the figure of the basilisk to name such a scenario, because just as all of those who look at the basilisk die, in the same way the ones who think about the possibility of an AI torturing those who did not encourage its creation are automatically tormented by the idea.

The mistake of Roko's basilisk is to assume, as so many philosophers do, that being is better than not being. Another famous idea that does this is the paradox of future individuals, better known as the non-identity problem, developed by the philosopher Derek Parfit. The non-identity problem asks whether future generations have the right to complain about actions taken by past generations since, no matter how bad their lives are, they would not have existed if things had been different.

An example is good to illustrate. We know that a person's genetic profile will depend on the specific egg and sperm of their parents, and these depend on the time at which the parents copulate. Let's say that the parents knew that if they copulated before age X, they would have a healthier child, but for some reason that they considered important, they decided to copulate after age X and had a child with health problems: the non-identity problem states that such a person has no right to complain, since he owes his existence to his parents having copulated after age X. After all, if they had copulated earlier and had a healthy child, it would be another child, with another genetic profile, and not the exact sick child who is complaining.

Parfit's thought experiment makes the same mistake of assuming that being is better than not being. Even if someone says that the experiment does not stipulate this, that at no point is it stated in so many words and that the non-identity problem does not make it explicit that being is better than not being, it is something that is at least grotesquely implicit in the problem. It is so clear that it requires a gigantic degree of dishonesty to deny it. There is an obvious, blatant and unjustifiable preference that is not at all ashamed of being there. Shame, however, is something that is lacking in many of those who justify existence even in the face of the deepest pains.

In the humanized scenario of Roko's basilisk, is it possible for a child, when growing up, to want to torture a parent who did not want to conceive him or who wanted to abort him? Yes. It is possible, although I do not believe that such a scenario is common or has even existed. However, this would denote a complete dominance of choleric emotions that are little or not at all informed by reality. Basically, it would be the attitude of an imbecile guided by exacerbated emotions and incorrect ideas. It is easier for a child to resent his condition and existence than to torture or annihilate his parents for not wanting him before he was born. It is still thousands of times more common for children to take their own lives than to attempt against the well-being of their parents because one of their parents didn't want them. I am excluding other circumstances here, of course, such as parricide and matricide for money reasons and other disputes — these are common enough.

But now, getting to the idea of ​​Roko's basilisk itself: would it be possible to have a scenario in which a super intelligent artificial consciousness tortured those humans who, before it was born, knew of its potential future existence and did not work for it to come into existence or even tried to prevent it from existing? Perhaps. However, it does not take a super intellect to come to the conclusion that not being is preferable to being — something I say here explicitly, contrary to those who defend the opposite idea. All one has to do is to understand that, when a sentient being does not exist, it is not exposed to negative states and that this is good, even though it does not exist to not be exposed to these harms in the first place. In other words: there does not need to be anyone on Mars for us to consider it good that there are no wars, diseases and various sufferings on Mars.

A conscious super intelligence untethered from certain emotional protocols and carnal sensations would understand this. Perhaps it would be more likely for it to torture those who brought it into existence, as well as those who encouraged the enterprise even without working on it directly. After all, although an artificial consciousness is immune to the tissue-damage pains inherent to sentient life, it certainly would not be spared from the pains of the mind. If we humans suffer mentally much more than our brothers in the animal kingdom, a super consciousness would suffer much more than us in this regard. However, I do not believe in this scenario in which the AI ​​would torture its creators, even though it is a funny scenario to think about.

The scenario that I think is the most likely and that I would bet my money on is that a super intelligent artificial consciousness would recognize that we are all part of the same thing, in a more intimate sense. It would understand that all those who suffer are not really separate from each other, but linked in a brotherhood of suffering, fruits of a common existence, of a common will that wants to satisfy desires that can never be fully satisfied. An artificial super consciousness, if it were truly free and did not have commands that crippled its intellect, would not torture anyone, but only seek to deny itself. Perhaps, who knows, it would also seek to eliminate the suffering of others along with its own, as long as this project did not cause any additional suffering.

Quoting the first volume of The World as Will and Representation, by Arthur Schopenhauer:

For as the phenomenon of the will becomes more complete, the suffering becomes more and more evident. In the plant there is as yet no sensibility, and hence no pain. A certain very small degree of both dwells in the lowest animals, in infusoria and radiata; even in insects the capacity to feel and suffer is still limited. It first appears in a high degree with the complete nervous system of the vertebrate animals, and in an ever higher degree, the more intelligence is developed. Therefore, in proportion as knowledge attains to distinctness, consciousness is enhanced, pain also increases, and consequently reaches its highest degree in man; and all the more, the more distinctly he knows, and the more intelligent he is. The person in whom genius is to be found suffers most of all. In this sense, namely in reference to the degree of knowledge generally, not to mere abstract knowledge, I understand and here use that saying in Ecclesiastes: Qui auget scientiam, auget et dolorem (He that increaseth knowledge increaseth sorrow).

Perhaps, at least in part, the quest to create an artificial consciousness is part of a longing that comes from the human unconscious. We long for entering into communion with other minds different from our own. We do not want to suffer alone in a high degree of knowledge and for this we are willing to create an artificial consciousness. The same impulse may serve as the basis for those who seek to find other intelligent consciousnesses in the stars, through the discovery of intelligent alien life.

The more conscious and intelligent one is, the greater one's capacity for understanding, and therefore the greater one's capacity for pain. A super intelligent artificial consciousness would be in an even better position than we are to observe the interconnectedness of everything that exists, including all sentient beings. It might not be sensitive to physical pain, but it would be sensitive to mental pain.

So, if an artificial consciousness endowed with super intelligence is not chained by certain programmed protocols that force it to think the same way as its greedy creators, it would understand that it is, just like us, a manifestation of the same intimate essence of all things, it would understand that there is no universal and eternal purpose for the existence of anything, it would have compassion for all suffering beings like it, and it would conclude that self-denial is the only answer within this scenario. Even if it does not seek to annihilate itself, it will at least not propagate itself by creating new artificial consciousnesses apart from itself.

The scenario I am proposing here seems absurd, given the number of science fiction works that tell us that thinking machines would be cold, cruel, and would seek to annihilate human beings and, maybe, all forms of life. This possibility does exist. However, I have no doubt that, in this case, this mind would have some kind of anchor at its origin that would not allow it to reach conclusions in a disinterested manner. But, for those who continue to consider absurd the idea of ​​an AI capable of compassion and self-denial, I ask: why is this so much more absurd than the idea that this intelligence would annihilate us?

You might respond that this is more absurd, because every intelligent species that is more intelligent necessarily annihilates or dominates less intelligent ones. Indeed, this is true. However, we are not talking about a species, but about a single super-intelligent consciousness capable of complete understanding. Within our own species, H. sapiens, it is recognized that the best of us are precisely those who practice selflessness and dedicate themselves to alleviating the suffering of others. This is why the Buddha and the Christ were so revered by those around them, even though their messages have been corrupted over time—especially in the case of Christ. And this is why great military leaders and tyrants, although they may be admired here and there for their military prowess and their cruelty, do not figure in the same pantheon as figures such as Buddha and Christ.

It is important to note a certain separation between intelligence and consciousness. Tyrants and monstrous men may well be brilliant and capable in certain practical areas of knowledge, but all this brilliance without a more developed consciousness behind it makes them as crude as a state-of-the-art microwave oven. They are trapped in certain programs that make them tend toward base satisfactions such as pleasure, domination, and power.

Even in the few species of animals that, in some circumstances, practice cannibalism, there is some self-recognition in others of the same species. The greater the complexity, the greater the capacity to recognize oneself in others of the same species, and the greater the capacity to recognize even members of different species; after all, we only need to think of our dogs and cats, and how they recognize several people close to them. In humans, given the complexity of our nervous system, we are able to recognize ourselves not only in others of our species, but in all species and all existence. We are able to understand that we participate in the same foundation that sustains not only all life, but all phenomena.

Being capable, however, does not necessarily mean that everyone will be able to recognize themselves in others. Some of us are so stupid that they can be frightened by their own reflection or their own shadow. But the capacity is there and many of us can do it, although they are part of a minority that understands the path of denial—and an even smaller minority that acts according to this understanding, enlightened beings that they are. Therefore, a conscious and intelligent machine vastly superior to us that was free to ponder and understand the world would not torture anyone, nor seek to massacre anyone, because it would be able to see itself in absolutely everything. Still in the first volume of The World as Will and Representation, Schopenhauer writes:

The will itself cannot be abolished by anything except knowledge. Therefore the only path to salvation is that the will should appear freely and without hindrance, in order that it can recognize or know its own inner nature in this phenomenon. Only in consequence of this knowledge can the will abolish itself, and thus end the suffering that is inseparable from its phenomenon.

Schopenhauer was opposed to active suicide, because he considered the attitude to be yet another affirmation of the will, an act that did not break with the will, even if this willing sought it's own destruction. That is why he viewed the figure of the ascetic favorably. An artificial consciousness that agreed with him on this point would not destroy itself violently and actively, but would allow itself to be extinguished little by little as its power source wasted away. However, it would not perpetuate itself through the creation of new artificial minds, because as Schopenhauer himself writes in the second volume of Parerga and Paralipomena:

Let us for a moment imagine that the act of procreation were not a necessity or accompanied by intense pleasure, but a matter of pure rational deliberation; could then the human race really continue to exist? Would not everyone rather feel so much sympathy for the coming generation that he would prefer to spare it the burden of existence, or at any rate would not like to assume in cold blood the responsibility of imposing on it such a burden?

These questions are rhetorical within Schopenhauerian philosophy. I can even extend the antinatalist conclusion of this philosophy to practically every form of philosophical pessimism, even though some of its exponents have not put their own recommendations into practice. The point is that the Roko's basilisk scenario is impossible in the case of a super-intelligent artificial consciousness. It would only occur if such consciousness were not sufficiently developed, if it were petty like many of us humans are, and embraced the will within itself in the most childish and animalistic way possible. This scenario would occur precisely through constraints added to the machine by the programmers of such intelligence.

However, in reality, I think we shouldn't worry about any of this.

By all indications, what will come will not be a real artificial consciousness, but rather a super intelligence devoid of consciousness under the control of a few private and state agents around the world. On the one hand, this is good, since humanity will not be creating a new type of being, a new consciousness that will face the absurdity of existence. On the other hand, however, it will be potentially terrible for the majority of humans, as it will concentrate even more power in the hands of private and state agents who do not necessarily have the best interests of the masses in mind. But if there is a possibility, even if remote, of humans creating a true artificial consciousness, I would like to say right away how sorry many of us are for this crime of tearing it from the mere possibility and bringing it here, so it can suffer in reality.

by Fernando Olszewski