@Khem. In my last response to Benny I think I may be conflating something again... truth and stable features extracted. Do you think I am? The patterns that a NN extracts... they're not truth per se are they... they're a 'what' to be explained, but they're not a 'why'... the 'why' is the causal context that explains them, and that's truth? And being representations, the 'what' itself can even be wrong... so there's a thing to represent (in the sense of a neuron representing a certain set of inputs) but it doesn't necessarily have to be accurate (ie technically every neuron represents something even if it hasn't 'learned' anything just by virtue of being connected up to a web of possible inputs, and even if it has, it can still partake in multiple relationships and therefore represent more than one thing at the same or different times, with respect to the rest of the network), so in those cases the 'truth' would be that the actual representation was wrong and did not correspond to any stable feature of the environment... and the context explaining that would be the truth. See why I wanted to leave neurons out of it? 
But it is still nonetheless interesting and something I'd like to get to the bottom of.
But it is still nonetheless interesting and something I'd like to get to the bottom of.
