Supermathematics and Artificial General Intelligence

[Edwardo Piet]

[ThoughtCurvature]

Mathilda, in contrast, research is going in directions largely concerning very general algorithms, or general intelligence.

Don't forget about unsupervised learning models that already exist today (and are only improving):

(1) Manifold learning or Deepmind's "Early Visual Concept Learning with Unsupervised Deep Learning"

(2) Generative Adversarial Networks that uses unsupervised learning (See Wikipedia "Generative_adversarial_networks")

I've been working on unsupervised learning models for over 20 years but thanks for telling me that they exist.

While I agree that unsupervised learning is a step in the right direction towards artificial general intelligence, or strong AI, there is a big difference between a AI that can learn a static data set unsupervised and one that can act, learn and relearn in real time. This is important because the real-world is not a static data set.

I am also well aware of the need for generalisation. As far as I am concerned, this is the key reason why so much AI fails. People get excited by initial results but don't realise that the challenge actually lays in scaling it up because by doing so the problem domain grows exponentially.

You also did not respond to my point that about EU's GDPR which demands that automated decision making needs to provide explanations which will dramatically reduce the profitability of the current deep learning approach. It is precisely because neural networks lack explanatory power that I have stopped using them myself.

Remember, we can see real time models, in unsupervised learning, that are already scaling well: (See: "Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks")

As for EU's GDPR regulation, I would have to research that a bit more before responding properly on that.
Deep learning models are in the regime of very very general algorithms, and this level of general application is only widening with time.

[I_am_not_mafia]

You do know what real-time means don't you? So far you've only given me papers that have unsupervised learning act on static data sets. And all of it concerns visual processing. What about action selection? Temporal sequence learning? Memory? Motor-co0rdination? etc

[LastPoet]

He's STILL not banned? Holy crap. If he stuck out anymore he'd erupt.

Remember what the apocalipse tanks in red alert 2 said? "Be patient".

[Alex K]

Making e.g. neural networks manifestly supersymmetric is at least an amusing idea. I don't know what it would give us, but hey.

[I_am_not_mafia]

Ugh! Neural networks are so 2016.

[causal code]

Ugh! Neural networks are so 2016.

Interesting topic. (I wanted to ask ThoughtCurvature some questions, but I see that isn't possible)

Anyway, neural nets are the most powerful learning algorithms used in 2017? (All I see at arXiv is neural nets)

Wouldn't neural nets be the best thing we can do, since the only example of general intelligence is biological brains, which neural nets are modelled on?

What would work better than neural nets?

[Fireball]

SOCKO!

[Whateverist]

Making e.g. neural networks manifestly supersymmetric is at least an amusing idea. I don't know what it would give us, but hey.

Maybe tidiness is next to godliness.

[I_am_not_mafia]

Ugh! Neural networks are so 2016.

Interesting topic. (I wanted to ask ThoughtCurvature some questions, but I see that isn't possible)

Anyway, neural nets are the most powerful learning algorithms used in 2017? (All I see at arXiv is neural nets)

Wouldn't neural nets be the best thing we can do, since the only example of general intelligence is biological brains, which neural nets are modelled on?

What would work better than neural nets?

Well I was joking when I said it, I was trying to imitate a fashionista.

But you can't say that neural networks are the best thing that we can do because the only example of general intelligence are biological brains.

Real-world neurons are completely different to what you find in artificial neural nets. Each one is far more complex and can do far more. They have dendritic trees, they adapt and habituate, they have local learning rules, they respond over different time scales to neuromodulators etc. Artificial neural networks were biologically inspired, they are not biologically plausible. That's not to say that we don't also have biologically plausible neural networks, my own ones are, but they haven't yet found to be as useful.

It is better to think of all the current learning models in AI as inherently mathematical, or computational. Neurons in artificial neural networks for example typically have activation functions for example. They perform statistical functions and used as such within a framework that controls them. If you consider it like this, then it is not too difficult to imagine the future of AI as also being mathematical. After all, we use Maths in many other sciences. Its power comes from being able to create abstract, concise and workable descriptions of reality. Its strengths are also useful for AI.

One major problem with neural networks though is it is difficult to explain their results. This becomes particularly important if it means that someone has been declined life insurance or a loan for example. But it is also important in that the better able we are to understand how or models are functioning, the better able we will be to develop new and better models.

What most people read about now as AI is just the latest hype cycle. There have been many other areas of AI in the past, some that also have been hyped up e.g. expert systems. There is also computational linguistics. I remember speaking to a colleague about 15 years ago who was a computational linguist and he didn't know that there was anything else in AI other than natural language processing until I reminded him that neural networks exist. Computational linguistics has always been inherently statistical.There's robotics and action selection with autonomous agents, which require different techniques than that used for machine learning / data mining which works on huge static datasets.

What you can do with a neural network you can also do with other techniques. Most of them fall out of favour though as people don't really know how to use them practically. Neural networks are connectionist systems. Not all connectionist systems are neural networks. I think that AI will always need a form of connectionism though because it models how complexity has developed in the real world.

I create self organising systems and used to use biologically plausible neural networks but hit a brick wall on how to scale them up and still understand what was going on. I've personally spent the last 7 years developing an alternative to neural networks that uses dynamical systems and I published initial results earlier this year, hence the joke in my previous post. I remember reading about dynamical systems 20 years ago but while it felt intuitively right, no one had any idea how to develop the idea. Neural networks can be understood sometimes as dynamical systems. There are many other techniques that have been developed in the past. Some such as cellular automata have found uses in other areas such as fluid dynamics. There's also evolutionary computation and artificial life. This has given us genetic algorithms which can be considered a search technique.

AI development will be shaped by the economic opportunities available in the future, but this does not mean that research in different applications of AI won't also continue, albeit at a slower pace.