Roko's Basilisk (Read Warning In Post First)

[GrandizerII]

I read RW too, and found the article on Roko's Basilisk. It's a strange tale of what ideology can do to a person's brain, modeled in a very strange, but very real, way. It's so bizarre to even imagine that we can actually reach a point in our AI (outside of a sci-fi story) that we can create a super-intelligent computer that can recreate a person exactly, which would be the exact same as you, and such a super-intelligence would like to do nothing more than torture this exact copy of you for not bringing it into being.

And watching Black Mirror has made this even more implausible for me, since (at least) three episodes include "cookies," which are exact copies of a person extracted by computer, which frequently get tortured, but even then, as they're perceived as having the same feelings as humans, even if the powers that be don't bother to treat them like it, even then there's still a distinction between what's happening to the original person and what's happening to the cookie.

Sadly, I can't find a video of the whole segment which properly contrasts what happened with Cookie!Greta and Real!Greta, so here's the clip of Cookie!Greta doing her work, already broken as Real!Greta goes  about her day.

And I think continuity (or at least as far as Planck's length allows!) plays a good role in the identity of self. When simulation/cloning occurs, there is a split there between the two selves so that one self goes along its own continuous path while the other on its own different continuous path in life. So they're not only two different bodies and brains, but reasonably two different selves.

The other thing I forgot to mention is that an AI as intelligent as the Basilisk should see no rational reason to torture simulated copies of the original beings who won't be affected by the torture in any tangible way. I think.

[Aoi Magi]

See if there is an "all-powerful" AI seeking to punish people, it doesn't really matter much whether you are on its good side or bad side, cause sooner or later you are gonna get spanked as well.

[GrandizerII]

See if there is an "all-powerful" AI seeking to punish people, it doesn't really matter much whether you are on its good side or bad side, cause sooner or later you are gonna get spanked as well.

If ... that's the key word. Experts in AI are hoping to avoid that from ever happening.

Plus ... there's spanking ... and then there's eternal torture. I'll have no qualms with just a bit of temporary spanking.

[Rev. Rye]

And I think continuity (or at least as far as Planck's length allows!) plays a good role in the identity of self. When simulation/cloning occurs, there is a split there between the two selves so that one self goes along its own continuous path while the other on its own different continuous path in life. So they're not only two different bodies and brains, but reasonably two different selves.

Which is a point made in the second episode that used cookies as a major plot point, U.S.S. Callister. Daly (or as I liked to call him, Captain Todd, for reasons that should be obvious to any Breaking Bad fans watching) cloned the protagonist of the episode and put the clone in her computer game. The clone finds out pretty damn quick that Daly's a monster who can torture people into taking part of his Star Trek fantasies, but the original knows nothing about the full extent of his fucked-up-ness, and her cookie only tells her the bare minimum of what's going on, so there's a distinct possibility that when Daly's brain is wiped at the end of the episode and he's pretty much dead, she'll probably still mourn him as her cookie (and those of her coworkers) celebrates their freedom.

The other thing I forgot to mention is that an AI as intelligent as that one should see no rational reason to torture simulated copies of beings who won't be affected by the torture in any tangible way. I think.

Yeah, the big question with such a plan: why would it even do such a thing? What would even be the point? Is it the sort of petty being who'd create voodoo dolls and stick pins in them in the hopes that someone who doesn't even know he exists will be hurt?

[tykobrian]

Not interested in shaping my entire life just so it may humour someone (something?)

[Little lunch]

I'm not worried about it as I know that I'm looking forward to having an AI master and would always do as I'm told.
In fact, my whole family and all my friends feel this way.

[I_am_not_mafia]

Aha! A thread about my favourite subject.

Roko's basilisk is a thought experiment about the potential risks involved in developing artificial intelligence. The premise is that an all-powerful artificial intelligence from the future could retroactively punish those who did not help bring about its existence, including those who merely knew about the possible development of such a being.

I have heard of this before. I didn't really pay much attention to it the first time I heard it. As with all the scaremongering it relies on advances in technology that we can't even guess at how it could possibly work. In this case it's time travel. With the singularity it's the idea that an AI could redesign itself without having to test each design. Because if it didn't have to do that then we've had the technology for the singularity since 1975 with genetic algorithms. Even further back if you include generic search techniques.

As far as I can see, it's all a just a new religion for modern day techies who prefer it to bronze age mythology.

I shall refer to your first quote which basically is worth repeating as many times as it takes.

The smartest people I know who do personally work on AI think the scaremongering coming from people who don't work on AI is lunacy.
—Marc Andreessen

Saying that, my husband repeatedly jokes (I assume) that his sole purpose in life is to stop me developing artificial intelligence that will bring about the downfall of civilisation.

[BrianSoddingBoru4]

Relax. *beep* Artificial intelligence is nothing you need be concerned about. *whirr click*

Boru

[I_am_not_mafia]

If there is a super intelligent AI of the future then I don't think it ever invents time travel. If it did, it could quite easily leave a few useful scientific papers on my door step to speed up the process of discovery.

Yet the only thing I find on my door step are malnourished human corpses with barcode tattoos and cauterised holes in the back of their heads.

[LastPoet]

You would sooner get cats ruling the would. The only thing that keeps us in check are dogs.