I shall try to demonstrate what I mean when I say that if you want your solution to be able to adapt to a wider variety of situations then it will by necessity be more complex. I really am trying to explain things here so please do not start to get angry with me which your latest post suggests.
Imagine that you are given the task of creating a black box which performs a specific function. Being a black box the user does not know how it works. They cannot see inside. All they see is a single input and the ten outputs. The input allows the user to tell the black box whether it has chosen the correct output. The reason why the user wants this black box is to plug into an external system. The blackbox chooses an action and the external system returns a measurement as to how successful that action was.
Any computer programmer will tell you that this is a trivial problem. You can come up with a solution that is efficient in terms of memory and processing power. So this is what the user chooses. They could have asked for a really complicated system using biologically plausible neural networks, or even real live cells, or some other completely self organising system. Both solutions adapt but the computer program below will be smaller, faster and more accurate than the complex system. There, well done, compared to using a live human, you've chosen something that performs better and more efficiently.
Here is some pseudo-code:
But now imagine the user takes their black box and plugs it into another application. Over time the system changes and the outputs initially chosen by the black box no longer correspond to the best choices. The black box is now useless for this task and needs to be reprogrammed.
So the user takes it back to the creator of the black box who add in a loop to periodically check all the actions before deciding on which is the best one. But how often should this check be made? The programmer decides to write a check to see whether the actions being executed bring in the same expected reward each time. But at what point should it start exploring alternatives rather than exploiting what it already does? The programmer examines the system the user wants to put it into and decides on a decent error threshold before the program needs to explore other actions.
The small and fast system has now become slightly more complicated, slightly slower and cannot now act optimally because it needs to explore alternatives. It also now requires some values to be configured in order to operate.
The user then happily plugs this black box back into their second system and it works well although not optimally.
Now say that the user plugs it into a third system. This system suffers from a lot of noise and the best action that the blackbox needs to choose changes at irregular intervals. The computer program sometimes works well and sometimes doesn't because of the values chosen by the programmer. The user then takes the black box back to the programmer who adds in a load of statistical techniques for filtering out noise and tracking average error thresholds over time. The programmer then needs to decide on even more values to hard code into the blackbox. Now the program has become significantly more complex in order for it to be able to adapt to a noisier more variable environment.
Now imagine the user plugs the black box into yet another system. This time the black box shouldn't decide on a single action but choose the relative strength of each output. The program is not up to this task at all and needs to be completely re-written from scratch. On the other hand, if the user had chosen the second black box to begin with, the one contained the evolved self organising neural network, then they could have re-used it each time without needing to take it back to the programmer.
Computers are very fast idiots. They follow explicit instructions. The reason for developing Artificial Intelligence is to allow them to adapt so we do not have to specify everything explicitly. At each stage the initial solution chosen by the user became more complex, required more values to be specified and in the end did not scale.
Imagine that you are given the task of creating a black box which performs a specific function. Being a black box the user does not know how it works. They cannot see inside. All they see is a single input and the ten outputs. The input allows the user to tell the black box whether it has chosen the correct output. The reason why the user wants this black box is to plug into an external system. The blackbox chooses an action and the external system returns a measurement as to how successful that action was.
Any computer programmer will tell you that this is a trivial problem. You can come up with a solution that is efficient in terms of memory and processing power. So this is what the user chooses. They could have asked for a really complicated system using biologically plausible neural networks, or even real live cells, or some other completely self organising system. Both solutions adapt but the computer program below will be smaller, faster and more accurate than the complex system. There, well done, compared to using a live human, you've chosen something that performs better and more efficiently.
Here is some pseudo-code:
But now imagine the user takes their black box and plugs it into another application. Over time the system changes and the outputs initially chosen by the black box no longer correspond to the best choices. The black box is now useless for this task and needs to be reprogrammed.
So the user takes it back to the creator of the black box who add in a loop to periodically check all the actions before deciding on which is the best one. But how often should this check be made? The programmer decides to write a check to see whether the actions being executed bring in the same expected reward each time. But at what point should it start exploring alternatives rather than exploiting what it already does? The programmer examines the system the user wants to put it into and decides on a decent error threshold before the program needs to explore other actions.
The small and fast system has now become slightly more complicated, slightly slower and cannot now act optimally because it needs to explore alternatives. It also now requires some values to be configured in order to operate.
The user then happily plugs this black box back into their second system and it works well although not optimally.
Now say that the user plugs it into a third system. This system suffers from a lot of noise and the best action that the blackbox needs to choose changes at irregular intervals. The computer program sometimes works well and sometimes doesn't because of the values chosen by the programmer. The user then takes the black box back to the programmer who adds in a load of statistical techniques for filtering out noise and tracking average error thresholds over time. The programmer then needs to decide on even more values to hard code into the blackbox. Now the program has become significantly more complex in order for it to be able to adapt to a noisier more variable environment.
Now imagine the user plugs the black box into yet another system. This time the black box shouldn't decide on a single action but choose the relative strength of each output. The program is not up to this task at all and needs to be completely re-written from scratch. On the other hand, if the user had chosen the second black box to begin with, the one contained the evolved self organising neural network, then they could have re-used it each time without needing to take it back to the programmer.
Computers are very fast idiots. They follow explicit instructions. The reason for developing Artificial Intelligence is to allow them to adapt so we do not have to specify everything explicitly. At each stage the initial solution chosen by the user became more complex, required more values to be specified and in the end did not scale.
