Completeness of stack. If you're under the impression that your runtime ends where the Docker container does, that will eventually bite you--it's very rare for my workstation to be running the same kernel version as my servers and I've been bitten multiple times in the past by kernel bugs due to differences in version, so even if I use Docker (which is rare) I run it through a known-good virtual machine that mirrors production as closely as possible.
(Vagrant also has a Docker provider, but I can't think of a good reason to use it.)
Docker is a fragmented mess network wise if you need to support developers on both windows and macOS. Also, docker requires hyper-v and admin privs on Windows. And you can't run VirtualBox with hyper-v active so you need to choose either of the two options. Granted, you could run Docker in virtualbox but it wouldn't be very clean.
I use Minikube with the VirtualBox driver, and it is the only way that I have been able to deal with my slightly unusual networking requirements and do local dev with Kubernetes.
VirtualBox lets you forward ports from within the VM to your localhost directly, bypassing iptables and any default routes.
Our network uses a "no-split-tunneling" VPN, so most of the Docker networking solutions are completely unusable for me.
Kubernetes fortunately provides easy ways to enumerate the services that you intended to expose (via ingress, or similar) so it's absolutely trivial to script forwarding every exposed service or ingress to the localhost IP. I still am editing /etc/hosts file if I ever need to use a host-based route, and I have some interesting issues with SSL certificates that sometimes did not have the server name that I expected on them, but for the most part this works great for me.
I am a Mac user and showed my coworker who is a Windows user, we tried to do the same thing on his machine and it was even easier because there is no notion of privileged ports below 1024. So, it works the same way but with one less workaround.
The whole Hyper-V thing with the default Docker for Windows stuff has put me off - is there a workaround to go back to using a VirtualBox VM again? I do a fair amount of development on Windows machines along with Mac/Linux so I need something that works consistently among all those platforms.
Vagrant is filling the void for some of those projects since it just works with no fuss on Mac/Windows/Linux without forcing me to use Hyper-V.
Absolutely, you can boot your preferred distribution for running Docker on[1] (using Vagrant even! setting the provider to VirtualBox), forward the port that Docker is running on (2376?) to your physical host/localhost[2], and set a few variables on the client machine to get your docker client talking to that ("remote") docker daemon[3]
Or, as tmzt mentioned, minikube and minishift will also let you set --vm-driver=virtualbox on start. Those are nice even if you don't want to use Kubernetes (but there are plenty of options.)
You can still run Docker Machine (part of the old Toolbox) for VirtualBox support. minikube/minishift can also be used as docker VMs and download up to date iso images automatically.
Vagrant is also used as an easy way to spin up development environments e.g. for embedded systems via virtualbox. I am not aware of a simple way to create a GUI with docker. With vagrant it is just one line: v.gui = true
Can you elaborate on your first use case - for what kind of embedded development do you use virtualbox and how?
Regarding your second point - you can either attach a shell for testing (e.g. I often build a Dockerfile first interactively via /bin/bash inside the container) or use the hammer (a web-UI) and connect your localhost to the docker network interface.
Data exchange between host and container is also simply done via bind mounts - which might be more elaborate in production however.
I'm a big advocate of Kubernetes and I don't know the answer to this question, for sure. If you want to simulate a machine that has a kernel of its own, and you need to be able to make that kernel version different from the kernel version on the host, you definitely need Vagrant instead of Docker for that.
My team has no such requirements and IMHO uses Vagrant solely because of inertia. We've always used Vagrant, it's what most people have installed on their machine, there is a Vagrant box with some of the moderately difficult to configure things already done, so we all can use the same configuration, like the Oracle Client libraries and the nginx frontend with a self-signed localhost certificate (required so your local development can talk to our auth server).
There's absolutely no reason we couldn't do the same thing with Docker. We just haven't.
I would argue that if you aren't using Packer or if writing your own customizations into Vagrantfile, you aren't really using Vagrant and it's a somewhat harmful black-box for us. Those steps are baked into the box file, not done in a Vagrantfile as provisioning steps, not able to be inspected inside of an Ansible playbook; so that knowledge of how to do these things could easily be lost and it would be a headache to reproduce. Packer is roughly what we need to make it better. For my team, Vagrant is just a thin wrapper over VirtualBox, so the team does not need to know that they are using VirtualBox.
The second reason you might want to use Vagrant instead of Docker is if some leadership in your org has declared that you still may not use Docker for anything. This is the case here; you may use Docker but not without a good reason and not without having your usage reviewed by a panel of experts on various subjects (it's the Design Review Board.)
We got our usage of Docker approved so that we can manage Jenkins via Helm. The kubernetes-plugin for Jenkins creates pods as build slaves, and when they complete their jobs they go away. You want your builds to run in a clean environment, you want your slaves to be disposable; pods are ephemeral, may group containers together, and they go away when they complete the job. That's just exactly what problem this tech was meant to solve.
DRB thought that was a great justification and it was approved. I am still the only place that I'm aware of across the entire institution where Docker is used in an approved way though.
They're often complimentary, for example using Vagrant to spin up multiple VMs to run a Docker swarm mode cluster.
Vagrant is also great for learning about clustering technology. In minutes you can have dozens of VMs running on a single machine.
That said if you don't have specific OS requirements, then http://labs.play-with-docker.com/ works well for simulating a multitude of machines.
Both Docker and Vagrant rock for the same reason: image distribution. You don't have to know anything about building a VM or container image to be able to benefit from the tools.
Of course they both make it a breeze to manage the respective runtime environment too.
So in my mind Vagrant is to VMs what Docker is to Containers. Of course the use cases overlap, nonetheless they both are indispensable tools.
It's great for complex apps with multiple dependencies, but if you plan on never needing full VMs likely not worth it.
I'm used to liberally using crontab, iptable rules, multiple languages, and not deploying separate containers / vms if I need something like Redis. For some of them I'd end up with 6-8 containers if I went that route.
