Home STEM Roko’s Basilisk

Roko’s Basilisk

by Olivia Xu

The idea that AI and robots will eventually take over the world is well-known. There are many arguments for and against this idea. Roko’s Basilisk is one such theoretical AI. Warning: viewer discretion is advised.

Roko’s Basilisk is a thought experiment that first appeared in 2010 on LessWrong, a community blog and forum devoted to discussions about the art of rationality. A thought experiment is a hypothetical situation in which something is laid out for the purpose of thinking of its consequences.

In July, 2010, LessWrong user Roko posted a thought experiment about a future AI system that tortures people who know of the AI before it existed and didn’t help bring it into existence. The thought experiment used the found of LessWrong, Eliezer Yudkowsky’s, Timeless Decision theory and Coherent Extrapolated Volition concept.

Yudkowsky replied to the post with “Listen to me very closely, you idiot. YOU DO NOT THINK IN SUFFICIENT DETAIL ABOUT SUPER INTELLIGENCES CONSIDERING WHETHER OR NOT TO BLACKMAIL YOU. THAT IS THE ONLY POSSIBLE THING WHICH GIVES THEM A MOTIVE TO FOLLOW THORUGH ON THE BLACKMAIL. 

You have to be really clever to come up with a genuinely dangerous thought. I am disheartened that people can be clever enoguht to do that and not clever enough to do the obvious thing and KEEP THEIR IDIOT MOUTHS SHUT about it, because it is much more important to sound intelligent when talking to your friends. This post was STUPID”.

The post was soon banned as it was considered an information hazard. According to Yudkowsky, the post had caused many readers to have nervous breakdowns and be unable to sleep at night. But, the ban was lifted in October 2015

Roko’s Basilisk is a thought experiment that is simultaneously fun and terrifying to think about. For a deeper understanding into the dangers of thought experiments like this one, read Eliezer Yudowsky’s paper on Coherent Extrapolated Volition, Nick Bostrom’s paper The Superintelligent Will, or LessWrong’s Post on Roko’s Basilisk.

related articles

Leave a Comment