[Serious] Is ChatGPT a Chinese Room?

[Angrboda]

What beyond coherency would you say is required for truth?

That's a tough one, but one thing I can say is that the experience of certainty is often wrong, and not a reliable guide at all to the truth. In the game of Mafia you can have it all 'figured out' in a way that 'fits together' and you can often experience a sense of certainty that goes to the core of your being... as you might remember, I think you've played in the past? But then the 'flip' comes, and overturns everything you thought you knew. Or another example would be lateral thinking puzzles... puzzles where the only way to solve them is to think outside the box of your existing assumptions, the 'box' in that sense being an internally coherent understanding itself. So what makes one or other understanding closer to the actual truth?

This is just off the top of my head, so please forgive it if it's not well thought-out but from these examples it seems to be more a matter of degree, ie in both cases new information comes in that needs to integrated, and so the new representation better models the available data than the old representation. So I guess from this point of view this would make me some sort of relativist regarding truth because the brain in this sense is doing the same thing all the time, trying it's best to accurately model reality, or at least inasmuch as it's required to do so to survive... integrating ever more and more information with its existing model. So with each new integration it perhaps gets closer to some 'objective' truth, as a measure of integration, coherency, and perhaps add quantity of data (ie a lifetime's worth of observations about the world), but how to define that outside of what the brain seems to be doing, I don't know. I'll have to think about that.

ETA: Forget this, it's not a great answer and relativist is probably not the right word.

I'm not looking for a finished answer, just asking your thoughts. I think it's natural to think that our approximations are getting closer and closer to the truth, but this presents several issues. The first is the question of how we know that we're getting closer to truth rather than further away. I recall a discussion I once had with an individual who, on the basis of the increasing sophistication of science and technology, we could predict that the upward trend would continue and that eventually most mysteries would be solved. But it turns out that such intuitions have no real foundation. If you picture increasing knowledge as a line on a graph trending upward, it's simply a truism that the line could change direction at any time unless we have independent reason to believe that it won't; predicting the future course of the graph's plot based solely on the fact that it has been trending upward consistently in the past is fallacious -- you can't assume the past behavior will continue. But the intuition is strong. Why do we feel confidence that our approximations are even close to the truth? I think the common answer is that if our approximations weren't close to true then they wouldn't be as useful as they are. I'm not sure that intuition is valid either, but I think it's a genuine question as to whether it is or not. Even if our approximations were getting less and less accurate, simply by virtue of continuous applied effort we'd find more and more useful things. In a sense, I'm suspicious that the belief that we are getting closer and closer to the truth in some sense depends upon the pretense of knowing where the truth lies. Like some theological questions, such as what God would do, answering the question presumes that we have the ability -- to independently gauge where truth lies -- when that is the very ability that is in question, just as answering what God would do might, in a strong sense, require that we be godlike in possessing omniscience and omnipotence and moral perfection like He does.