I think that the role that human "instincts" play in the development of intelligence in our brain is very important. Social behavior is a key part of that.
We do see high levels of intelligence in non-mammal species, like crows, but they tend to also be very social creatures. The main counter example I can think of would be the octopus.
I think that even if we crack "general intelligence" and can make something that can problem-solve and learn on par with an Octopus, that approach will not get us to human level cognition.
I personally do believe that you will need societies of AI agents to develop the culture software to achieve human level cognition. I think we greatly underestimate the value and complexity of the cultural OS's that allow humans to perform advanced cognition.
The book “Sapiens” does a great job of exploring how culture enabled us to evolve apart from changes to our DNA. I couldn’t agree more.
On instincts: the neocortex is built on top of all the other more instinctual parts of the brain (vision, movement, etc). But this may just be an artifact of being embodied. Is there much difference between a RL agent that gets its data by processing pixels or directly from a simulator? I wouldn’t say there is, except that these non-pixel-parsing agents would have a hard time doing anything outside a simulator that was doing the processing for them.
You might really enjoy the science fiction novels Children of Time and Children of Ruin, if you haven’t already. Or hell, based on your comments, maybe you wrote them! Either way, they are spectacular and thought-provoking.
We do see high levels of intelligence in non-mammal species, like crows, but they tend to also be very social creatures. The main counter example I can think of would be the octopus.
I think that even if we crack "general intelligence" and can make something that can problem-solve and learn on par with an Octopus, that approach will not get us to human level cognition.
I personally do believe that you will need societies of AI agents to develop the culture software to achieve human level cognition. I think we greatly underestimate the value and complexity of the cultural OS's that allow humans to perform advanced cognition.