When people have multiple servers, often they need/want those servers to communicate. For example, I might have a web server and a separate database server. So anything on my web server that needs to make a database query would have to communicate over the network to the database server. Prior to this change, Digital Ocean users would have to do that communication over the public internet interface (public IPs), and the traffic would count towards their quota. Now, the servers can have private-side interfaces that aren't internet-facing, allowing this internal communication to take place on the internal network, avoiding the public interfaces and not counting towards the traffic quotas.
the latency will be similar in un-congested conditions (it will stay inside the network edge) but private networks allow the switched network to broadcast less, allowing it to handle "more" traffic across switch backplanes and area networks.
Well, the nature of routing is such that, for communication between two VPS in a given datacenter, the traffic wouldn't really "go over the internet", but it ought to still help to have the internal traffic separated from the public side.
