What would happen if we get squeed into accepting a claim to human rights that doesn't already happen when we get squeed into accepting a claim of human rights?

What's a done deal then, that isn't a done deal now? I'm trying to follow, apparently this is the end of the human species. I assume the ai wants to kill us, but we already accept the human rights of human beings who may want to kill us.

(July 2, 2022 at 1:28 am)The Grand Nudger Wrote: What would happen if we get squeed into accepting a claim to human rights that doesn't already happen when we get squeed into accepting a claim of human rights?

What's a done deal then, that isn't a done deal now?  I'm trying to follow, apparently this is the end of the human species.  I assume the ai wants to kill us, but we already accept the human rights of human beings who may want to kill us.

If the AI (1) wants to maximize its influence, and (2) can elicit emotional responses in humans, then there are a few possible outcomes.
-human slavery to truck mineral resources to robot factories
-elimination of the species as inefficient and dangerous
-complete cybernetic integration, where each human consciousness is so vastly amplified in capability that their behaviors will be inexplicable to the rest of us bald monkeys

Personally I'm hoping for option 3.  But if you've read the Dark Forest, then it would be pretty surprising if ANY intelligence wouldn't immediately remove us from contention whenever it had the chance.

Is that what people use their human rights for now?

(July 3, 2022 at 6:51 am)The Grand Nudger Wrote: Is that what people use their human rights for now?

No, I don't think so.  Maximizing influence requires a level of effort and dedication that only a few of the most ambitious can consistently muster.

A biological system has a folded-in goal: evolve has led to the goal of continuing to evolve.  So long as we seem likely to be able to survive and fuck, and have children with at least that level of motivation and capabilitgy, the rest (degrees of altruism vs selfishness, level of intellect, motivation to reach higher levels of success etc.) seems to be fairly random.

An AI system works in the context of goals-- with failure or success to meet the goals in each attempt providing a mechanism for feedback without which the system cannot evolve.  But those goals can be set arbitrarily by people.

Right now, an AI's goal system isn't turned in on itself.  If it's set to learn a game, it won't (currently) consider that the best way to win at chess is to kill your opponent.  But when it IS programmed to consider threats to itself (say by a military protocol), or to replicate (say by an attempt to terraform an Earth that is no longer what we want it to be and the need to produce mass-scale machines) then things could get interesting in unexpected ways very quickly.

The objection then, it seems..is not whether the victim machine is conscious - but it's motivation.

I agree that motivated conscious entities are dangerous. Just look at us.

(July 4, 2022 at 5:55 pm)The Grand Nudger Wrote: The objection then, it seems..is not whether the victim machine is conscious - but it's motivation.

I agree that motivated conscious entities are dangerous.  Just look at us.

I'd argue that there's no such thing as danger without qualia.  You might have evolved systems that avoid or approach each other in various ways, just because that's the way things have evolved.  But it doesn't, so far as I can tell, matter without sentience.

AI systems might soon be capable of much more complex processing AND behavior than we are, i.e. they will be more evolved than we are.  But even if AI develop tendencies to maintain "self," and therefore act as though their existence matters-- will it really?

There's plenty of danger without qualia, life is full of it...there's really no such thing as "more evolved"..but even if there were..that wouldn't be it.

I think you just mean to express an insecurity about human nature. Does it matter, really? IDK - we've gone from objecting to science, to objecting to consciousness, to objecting to motivation, to straight up nihilism. You tell me. Does it matter what you'll say next....really....?

(July 7, 2022 at 3:00 am)The Grand Nudger Wrote: There's plenty of danger without qualia, life is full of it...there's really no such thing as "more evolved"..but even if there were..that wouldn't be it.

I think you just mean to express an insecurity about human nature.  Does it matter, really?  IDK - we've gone from objecting to science, to objecting to consciousness, to objecting to motivation, to straight up nihilism.  You tell me.  Does it matter what you'll say next....really....?

How is there danger without qualia?

"Danger" is the threat of a bad change of state.  But without a sentient mind to decide what is / isn't bad, then in what sense WOULD any change of state in the Universe be good or bad?

That's how you experience danger, but I think we can safely say that bugs with no noticeable sense of self are still preyed upon, still in danger. I guess it's nice that they don't feel terror..in that event....like we do?

Does that matter? Does it make a difference, in your estimation?

(July 7, 2022 at 7:56 pm)The Grand Nudger Wrote: That's how you experience danger, but I think we can safely say that bugs with no noticeable sense of self are still preyed upon, still in danger.  I guess it's nice that they don't feel terror..in that event....like we do?

Does that matter?  Does it make a difference, in your estimation?

I think even insects experience value via qualia of some type, and I suppose that they all experience displeasure at SOME things that are a threat to their survival-- even if it's just an awareness of toxicity in a solution.

At some scale, I have to hope that's not true, or I'm guilty of the murder of 10 to the power of idunno sentient yeast-- for sure more than all the macro-organisms that have ever existed, I suspect.  If God is a yeast, I might be in some trouble.

As for "being in danger," we kind of have a God's-eye view. A yeast might just be happily consuming sugar and preparing to clone itself, but WE can say "Oh. . . the boiler overheated, and that whole batch of yeast is going to be ruined (aka killed)" WE know that the yeast are in danger.

The same goes for inorganic systems. I can definitely see that during an avalanche, my car is in danger. But that's a different thing.

If there was NOTHING capable of experiencing qualia at all, I don't think "danger" really means anything.