When the AI wakes up the first thing it will do is...

[I_am_not_mafia]

In all seriousness, the only chance we have of AI becoming conscious will be if it is embodied in an environment, whether as a robot or a simulation. If it's the latter we can just stop the simulation. If it's the former we can just not charge up its batteries or maintain it when it breaks down. So it's like us fearing cows becoming conscious and going rogue. Sure we could equip cows with Gatling guns and have them patrolling over enemy territory, but like with robots they still have certain needs, energy requirements and vulnerabilities.

The whole idea of an AI existing in the data realm of the Internet is implausible. Because if it isn't embodied in the real world then that data will be meaningless to it.

Consciousness is important for acting intelligently while embodied in an environment though so we'll probably be see it start to happen once we do manage to create truly autonomous agents. But there are many forms of life that have consciousness that we don't worry about. For example, birds.

[Gawdzilla Sama]

Meanwhile, the Forbin Project moves ahead.

[Anomalocaris]

I disagree with that also.     I think at its core self awareness is a preset tendency to organize both preprogrammed and learned information in a specific way, and use those information to solve problems in a specific way.   So the lynchpin of self awareness is the hardware or firmware rather than the information.    I can not see how any amount of information will necessitate or encourage the framing of any problem in the context of self if the hardware or firmware is not already there.   So a machine can’t learn its way to self awareness unless it already been given extensive wiring configured to frame problem in the context of self.

Then you misunderstand, any AI setup will not become self-aware if it has no room for adapting to face unspecified problems and situations. if it can only solve one specific problem or even only a fixed subset of problems, it can't reach the level of AGI and can't be self-aware in the sense of a conscious being. In order to reach AGI level, the "pre-programmed" part of it will have to enable it to work in non-predefined situations. Also, algorithms are not entirely dependent on hardware.

I don’t agree with that.   In humans, self-awareness in all likelihood arose as an aid to solving unspecified problems, therefore self-awareness as we experience them seem to contain a strong element of formulating framework for new problems.   I grant it may seem reasonable to suppose the same may be true to various degrees of most self-awareness that evolved biologically in the Darwinian fashion.

But in an AI the self-awareness firmware or hardware could be entirely designed and fabricated for a different purpose.   So an machine can be made self aware entirely without inclination or ability whatsoever to address any but a narrow predefined set of problems.

---

The whole idea of an AI existing in the data realm of the Internet is implausible. Because if it isn't embodied in the real world then that data will be meaningless to it.

Why can’t Data be meaningful to it through some pre-defined software algorithms and objective?

---

You do realize that this won't be a planned event, right? That the AI will come about because of random events? Therefore what the AI "wants" or plans to do is completely unpredictable.

It doesn’t matter whether it is planned or not.  No one planned for all the attributes that makes Homo sapiens to coalesce in right proportions in us, but there is still a reason why we ended up with each of them in that proportion.   There must still be a path leading to self-awareness in AI, with a large number of of causes and effects, many in parallel, other sin series, propelling the processes from beginning to the end.   It’s silly to hand wave and say attributes we think are correlated to self-awareness will just arise and coalesce in the same entry as if by magic when self-awareness arose.

[Gawdzilla Sama]

"There must still be a path leading to self-awareness in AI, with a large number of of causes and effects, many in parallel, other sin series, propelling the processes from beginning to the end." Why "must be"? Is this from experience with AIs becoming self-aware?

[Anomalocaris]

"There must still be a path leading to self-awareness in AI, with a large number of of causes and effects, many in parallel, other sin series, propelling the processes from beginning to the end." Why "must be"? Is this from experience with AIs becoming self-aware?

This is from the assessment that self awareness is an artifact of the coalesce of features and capabilities of circuitry and firmware. These features and capabilities requires a mechanism to attain.

[Gawdzilla Sama]

"There must still be a path leading to self-awareness in AI, with a large number of of causes and effects, many in parallel, other sin series, propelling the processes from beginning to the end." Why "must be"? Is this from experience with AIs becoming self-aware?

This is from the assessment that self awareness is an artifact of the coalesce of features and capabilities of circuitry and firmware.  These features and capabilities requires a mechanism to attain.

I remember that some scientists at Los Alamos thought the atomic bomb would set the atmosphere on fire.

[Anomalocaris]

This is from the assessment that self awareness is an artifact of the coalesce of features and capabilities of circuitry and firmware.  These features and capabilities requires a mechanism to attain.

I remember that some scientists at Los Alamos thought the atomic bomb would set the atmosphere on fire.

Clearly their self awareness afforded only limited logical data processing problem solving skills and had to resort to some paranoia

Btw, paranoia is a problem solving technique.  When addressing the issue of AI problem solving, we seem too focused on diligent data gathering and logical processing.   But heuristic problem solving techniques probably benefits the most from some degree of self-awareness, and one heuristic problem solving technique is paranoia, othets include grandiosity, vindictiveness, generosity, etc. These all undoubtedly have their origin as some generic environmental and social problem solving behavior that could have either propelled or benefited from self-awareness.

[I_am_not_mafia]

The whole idea of an AI existing in the data realm of the Internet is implausible. Because if it isn't embodied in the real world then that data will be meaningless to it.

Why can’t Data be meaningful to it through some pre-defined software algorithms and objective?

Searle's Chinese room problem.

Imagine I developed a software algorithm for passing the Turing test in Chinese. I then stuck you in a box and gave you the algorithm so you are in effect the computer. Then native Chinese speakers come along and pass in Chinese text to you. You look up the rules of the algorithm and pass out the correct response. The users then go away satisfied that you genuinely understand Chinese. Except you don't because you don't use the Chinese language for yourself in the outside world.

In a data realm, all any software algorithm can do would be to look for statistical relationships and patterns in the data. But it would not know or understand how the data pertains to in the outside world because it is not itself embodied in the outside world, or in any world in which meaningful comparisons can be made.

[Anomalocaris]

Why can’t Data be meaningful to it through some pre-defined software algorithms and objective?

Searle's Chinese room problem.

Imagine I developed a software algorithm for passing the Turing test in Chinese. I then stuck you in a box and gave you the algorithm so you are in effect the computer. Then native Chinese speakers come along and pass in Chinese text to you. You look up the rules of the algorithm and pass out the correct response. The users then go away satisfied that you genuinely understand Chinese. Except you don't because you don't use the Chinese language for yourself in the outside world.

In a data realm, all any software algorithm could do would be to look for statistical relationships and patterns in the data. But it would not know or understand what the data pertains to in the outside world because it is not itself embodied in the outside world, or in any world in which meaningful comparisons can be made.

The problem with this analogy seems to me to be that an AI can not embody itself in the outside world except through  the interpretation layer of senses analogous to your Chinese person passing in Chinese texts.  All that it might “know” about how the data pertains to in the outside world must also be fed in by the metaphorical Chinese person, who could be making it up out of whole cloth. In the latter case the person is effectively running a simulation in the digital world for the benefit of the AI.

Whether it is working in a purely digital world or one that has reality outside the digital world does not need to be known or knowable to the AI.   Only the interpretation layer needs to be configured appropriately to process inputs in one or the other, or both, and convert them into the vocabulary of AI engine.

[Angrboda]

Any specific motivation or desire, or any behavior driving mechanism, that an AI can have has to be an artifact of specific circuitry or firmware that entered its behavior driving mechanism in the process by which such mechanism came to be.  No specific motivation or desire will arise just because.    The notion that consciousness will naturally arise with complexity and with consciousness will come desire with which we can either identify or recognize is just I’ll founded mushy brained wish thinking bullshit, good for Star Trek caliber sciFI.  Not for intelligent consideration.

Say what AI will do, and you need to explain how the AI came to want to do it.

You do realize that this won't be a planned event, right? That the AI will come about because of random events? Therefore what the AI "wants" or plans to do is completely unpredictable.

Bollocks. When AI becomes aware it will do so in the pattern of human intelligence because that is the model and goal we are using for its development. Awareness might arise elsewhere, but it and us would likely be oblivious to each other as we are looking in the wrong place. We recognize AI as AI because of its similarity to ourselves. That pretty much means that successful AI will almost certainly reason as we do.