We re-implemented Vagrant's Chef provisioning feature on top of OpenSSH with ControlMaster. This cut about 3 mystery seconds mystery time from our provision process.

Instead of targeting local Virtualbox VMs, we use AWS boxes created by the internal tool we use to manage our production fleet.