(September 6, 2017 at 5:34 pm)causal code Wrote:(September 5, 2017 at 11:12 am)Mathilda Wrote: Ugh! Neural networks are so 2016.
Interesting topic. (I wanted to ask ThoughtCurvature some questions, but I see that isn't possible)
Anyway, neural nets are the most powerful learning algorithms used in 2017? (All I see at arXiv is neural nets)
Wouldn't neural nets be the best thing we can do, since the only example of general intelligence is biological brains, which neural nets are modelled on?
What would work better than neural nets?
Well I was joking when I said it, I was trying to imitate a fashionista.
But you can't say that neural networks are the best thing that we can do because the only example of general intelligence are biological brains.
Real-world neurons are completely different to what you find in artificial neural nets. Each one is far more complex and can do far more. They have dendritic trees, they adapt and habituate, they have local learning rules, they respond over different time scales to neuromodulators etc. Artificial neural networks were biologically inspired, they are not biologically plausible. That's not to say that we don't also have biologically plausible neural networks, my own ones are, but they haven't yet found to be as useful.
It is better to think of all the current learning models in AI as inherently mathematical, or computational. Neurons in artificial neural networks for example typically have activation functions for example. They perform statistical functions and used as such within a framework that controls them. If you consider it like this, then it is not too difficult to imagine the future of AI as also being mathematical. After all, we use Maths in many other sciences. Its power comes from being able to create abstract, concise and workable descriptions of reality. Its strengths are also useful for AI.
One major problem with neural networks though is it is difficult to explain their results. This becomes particularly important if it means that someone has been declined life insurance or a loan for example. But it is also important in that the better able we are to understand how or models are functioning, the better able we will be to develop new and better models.
What most people read about now as AI is just the latest hype cycle. There have been many other areas of AI in the past, some that also have been hyped up e.g. expert systems. There is also computational linguistics. I remember speaking to a colleague about 15 years ago who was a computational linguist and he didn't know that there was anything else in AI other than natural language processing until I reminded him that neural networks exist. Computational linguistics has always been inherently statistical.There's robotics and action selection with autonomous agents, which require different techniques than that used for machine learning / data mining which works on huge static datasets.
What you can do with a neural network you can also do with other techniques. Most of them fall out of favour though as people don't really know how to use them practically. Neural networks are connectionist systems. Not all connectionist systems are neural networks. I think that AI will always need a form of connectionism though because it models how complexity has developed in the real world.
I create self organising systems and used to use biologically plausible neural networks but hit a brick wall on how to scale them up and still understand what was going on. I've personally spent the last 7 years developing an alternative to neural networks that uses dynamical systems and I published initial results earlier this year, hence the joke in my previous post. I remember reading about dynamical systems 20 years ago but while it felt intuitively right, no one had any idea how to develop the idea. Neural networks can be understood sometimes as dynamical systems. There are many other techniques that have been developed in the past. Some such as cellular automata have found uses in other areas such as fluid dynamics. There's also evolutionary computation and artificial life. This has given us genetic algorithms which can be considered a search technique.
AI development will be shaped by the economic opportunities available in the future, but this does not mean that research in different applications of AI won't also continue, albeit at a slower pace.
