It seems that what you are saying is that current DL architectures are not efficient, and that similar performances could be achieved with less parameters (or neurons). You compare it to biological brains, which are insane learners but still has some pre-wired architectures that are optimized for the real world.

That does not really disprove the idea that DL can achieve AGI given the right protocol. Also, your view of brains is a bit idealized: an insane amount of training data is required (every sensors of the body providing a constant stream of data), and even with that, it is not clear that generalization, in the sense you propose it, can be achieved. You won't be able to recognize a bike the first time ever you see one. The model of a bike is not something the brain is born with. In the other hand, huge DL models such as GPT or DALL-E does achieve some human-like generalization.

Also, you reject the idea of having latent representations of the world stored inside the model, while biological brains (including bees) exactly do that. Proposing an alternative (if one exists) would be interesting.

Another remark, when pursuing AGI, using classifiers as examples of DL algorithms is a bit dishonest. AGI could need to be able to have some incidence in the real world, and to aknowledge the consequences of its actions during training, more like in RL.

Overally I find your essay interesting, but you are stating a lot of assumptions without strong arguments to back them up.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Adrien Pavao

Research engineer and PhD student in Machine Learning @ Université Paris-Saclay