I think a combination of cognitive architecture and other techniques from the symbolic era of AI are going to needed to get us to the next big step. It seems unlikely that training larger and larger models packed full of hidden layers can get all the way to AGI (or whatever you want to call the next evolution.)

I think only by mimicking the natural cognition and respecting its physical constraints we'll get there.

There are some pieces of "biologically-inspired" mechanisms here and there in current connectionist methods, like convolutional nets based on Neocognitron which was in turn based on visual cortex research of the 80s. And this is probably why some these models work well in their narrow domain.

Or in cognitive architectures like Soar or Act which borrow some pieces from cognitive psychology and memory research. But IMO they are too model-heavy and this is probably the reason they never caught up or even worked as well as connectionist systems.

Being simply "inspired" by nature is probably not enough, we'd have to really understand how the animal brain works at a system level to get to actual AI.

Kind of like figuring out the law of gravity from a fall of an apple.

>But IMO they are too model-heavy and this is probably the reason they never caught up or even worked as well as connectionist systems.

there are countless of model based / symbolic intelligent systems at work right now. I'd go so far as say that there is no fully complex 'end-to-end' learning system in place anywhere in the outside world.

Self-driving cars say use learning systems for perception but they still deploy human expertise, routing algorithms, the software at an engineering level is a designed architecture, and so on. I don't think any autonomous car has actually learned speed limits which would be a rather risky endeavor.