That's a social problem we'll have to address as remote working becomes ubiquitous. It helps if the remote environment is more immersive, with the team aware of each others' presence and activity even when they're not directly communicating. Keeps your head in the space, so to speak.
I think you just described somewhat of a dystopian future... :)
Most work is a group effort. Group efforts require the feeling of belonging to a group to be effective. You just cannot get that if you work remotely. Real human interaction cannot be faked or successfully replaced by electronic means.
Working remotely a couple of days a week is probably beneficial. It alleviates the pressure of constant interactions preventing some tasks from being accomplished. Especially for us technical folk, where the dominant personality is not one that draws energy from other people. We still need to interact though.
" Group efforts require the feeling of belonging to a group to be effective. You just cannot get that if you work remotely"
I wonder if someone told all the (paid, for the sake of argument)devs working on Open Source projects that they can't have the feeling of being part of a group since they are scattered all over the world.
Periodic face time is important. But even in the same building I rarely speak face-to-face with everybody. A message or call or video share is normal even for 'local workers'.
Remote workers can be very effective with the right tools. Hundreds of companies are switching as we speak. I work at Sococo, and we enable it! Some of our clients have actually closed their offices and gone entirely remote, and they love it.