[Serious] Is ChatGPT a Chinese Room?

[emjay]

If we ignore phenomenal consciousness altogether (and so discard imagery and visualizing and all that), then what does it really mean to understand the meaning of a word or statement? When I'm focusing on a word on this screen, what exactly goes on in my head that constitutes the interpretation of it in a sense that's categorically different from how a computer treats words? Is it just a matter of structural differences? Complexity? Or is there something more to the distinction?

This stuff is challenging to me.

The way I've always looked at this question is to think of what we call 'understanding', whether phenomenally or neurally, as the measure of an idea/concept's integration and coherency. Coherency though is not the same thing as correctness, so you can be wrong in your understanding of something, because it's not a measure of truth but rather of how well things fit together, rightly or wrongly. So to me, to "understand" something is to be in a non-questioning state regarding that thing, but that understanding can still be wrong (as any experience of 'certainty' in the game of Mafia frequently attests ).

And from a neural network point of view, I would (at least in the past, when I was more interested in neural networks) liken that same situation to being a measure of the 'settled-ness' of a neural network into a stable state. That is to say that for any set of information a NN is representing, it goes through a process of values fluctuating until it 'settles' into a stable state, and it stays in that state until new information (eg the addition, subtraction, or change of an input) needs to be integrated, at which point it fluctuates again until it reaches a new stable state. So from this point of view, a coherent and integrated state... a state of non-questioning... would be the settled state of the network and a questioning state, would be the state the network is in when new information is added that has yet to be integrated with existing information, with conscious questioning itself interrupting/slowing this process down, in the sense that it may require real-time action, such a searching for something or whatever, to find the missing links as it were, that integrate the new data with the old.

And all this from the perspective that everything differentiable in phenomenal consciousness, ie noticeable, has a corresponding neural representation (hence my view of determinism regarding this) not least because everything we can notice we can name, and to name means to associate with.

Ultimately though, I think this is probably naive. It's how I used to view it, and in these terms it's an understanding, in the sense that it's, to me at least, internally coherent and integrated, but whether it's the objective truth is a completely different matter and I think probably not because it is not grounded enough in reality, just extrapolated from a theoretical understanding of neural networks (ie from the sort of perspective of the book I used to love "Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain"... and to be clear, I'm not saying this book puts forward my theories above, it doesn't, just that this book was the foundation for those theories).