I'm not knowledgeable on communication between VMs and how to best restrict communication there, but I have tried to make my docker networks more secure.
I went a bit overkill for my reverse proxy and all the docker networks it's connected to. For each service I want to expose through my reverse proxy, I manage a network specifically for that service in my caddy docker compose file. I then refer to that external network in my servjce's docker compose file, so that caddy can access it. For example, caddy is on caddy_net-grafana and on caddy_net-homepage. Grafana and homepage are on those networks respectively. So with this setup, caddy can talk to Grafana and homepage, but Grafana and homepage cannot talk to each other.
It wasn't too bad to setup. I made my own conventions for keeping it manageable and it works for me. I did run into the problem where I had to increase the default subnet pool, as after you create like 30 or 31 networks there aren't any subnets left to give out to new docker networks.
Yup. It works pretty well for me. Just will likely have to increase the default address pool in daemon.json if you go all out.