Booting VMs is kind of the main one.

Booting VMs on your worker seems like exactly the kind of debris concourse is encouraging the avoidance of by enforcing the containerisation of your builds. That said, I can't think of a reason why you couldn't boot a VM in a privileged Garden container, have you tried?

Yeah - I can't remember the actual failure, but it did not work out well.

We also need the VMs to survive across build steps.

We are working on an P/IaaS - so integration testing requires actually booting a full stack, then running an install, then testing the resultant servers are actually running Kubernetes

Ah that's a shame, I still suspect it would _probably_ work with the right tweaking but I appreciate there's a hurdle there that might be frustrating to overcome and that's no good.

We also test a PaaS and require VM spin up but we call out to GCP/AWS as necessary.

> We also need the VMs to survive across build steps.

You might represent them as a pool resource, so they can be safely handed from job to job. A lot of teams do this for stuff where they want to share an expensive, long-lived, stateful resource.

Disclosure: I work at Pivotal.

The VMs are per job - once the CI run tests that the orchestration has created the right system config on the VMs, and that basic integration tests work, we destroy the VMs.

I have worked in two companies now with Cloud Foundry distros, and both of them have started using Concourse, and then moved to Jenkins (for non Cloud Foundry projects - I shudder to think what would happen to a distro that attempted to not use Concourse)

I'm still lost. You said at the top that you need the VMs to survive across build steps (I read that as jobs), but now they're per-job?

The Concourse->Jenkins thing will definitely happen without better docs, examples and widespread understanding. Concourse is non-obvious to newbies and the error messages need a lot of love. Or any love.

I don't quite understand the parenthetical, could you elaborate?

I second what wlamartin said. We have pipelines that do integration tests that spin up 30 instance clusters in AWS. Trying to spin up something similar inside a concourse worker just isn't worth trying.