Hello, I'm one of the founders of HashiCorp! We're super excited about this. I'm really proud of the team that got us to this point, thankful to our communities, and just generally excited about the future for us.
The project that really kicked off HashiCorp was Vagrant, and I "launched" it right here on HN over 8 years ago: https://news.ycombinator.com/item?id=1175901 So its always fun to see things come back around. :) For those that are less familiar these days, HashiCorp now has many major OSS projects: Vagrant, Packer, Consul, Terraform, Vault, Nomad (in addition to lots of minor ones and libraries) addressing different areas.
For the HN crowd, there are a couple background viewpoints that I would also like to share that may be helpful on painting the backdrop on the product and business behind HashiCorp. A first quick one is my tweet on the four dimensions for "multi-cloud": https://twitter.com/mitchellh/status/1022162653618135040 At HashiCorp we're mostly looking at enabling #1 (workflow portability), though that has downstream effects of helping with the other dimensions as well.
While I'm happy to answer any questions, this post is rather late for me local time (past 10:30 PM here) and I'm leaving tomorrow to get married this weekend so I can't promise anything!
I would love to give Terraform a spin. However one thing our team needs is support for Ansible. If you look at https://github.com/hashicorp/terraform/issues/2661, radekg has worked on an Ansible Provisioner, but it looks like not many people know about it. Could some of your team maybe look at the following pull request https://github.com/hashicorp/terraform/pull/19021 and help radekq to ship the Ansible Provisioner? Thanks!
Hey, thanks for the shout-out! I'm doing what I can to make the provisioner work as reasonably as possible. It would be awesome to get it into the core, but indeed, there's no pick up from Hashicorp. It might be that Ansible is just too wide of a scope to settle on one man's implementation, there are so many different workflows for Ansible, some of which going way past the Terraform provisioner.
It's easy to imagine that merging my PR would open a Pandora's box, leading to people requesting full Ansible support, without really thinking through if they make sense in the context of Terraform.
The one thing that would make sense for Hashicorp, is to figure out how to distribute provisioners, similar to https://registry.terraform.io for modules. Distributing provisioners faces the following problems:
Thanks for the contribution. So we definitely want to get Ansible in and there is a lot of pressure to do so. At this time the focus is 100% on 0.12 core and then we're going to take a look at provisioners again. Sorry for the delay.
Congratulations! I've been a fan of Vagrant for a long time and have done some Vault as well. I haven't yet had a chance to try Terraform, but everyone says it is better than Cloudformation. :-)
I'm curious what you think of this crazy idea?:
With databases, some ecosystems use declarative migrations, where you keep a description of your schema, and the system automatically takes you from here to there. I've heard .Net people say how nice this is. On the other hand with Rails & Django you have delta scripts that give imperative directions for getting from here to there, rather than a declaration of the end state. I like that solution a lot, because sometimes you need more control over what to do. (For example, changing a one-to-many relationship to many-to-many, and moving the existing data.)
It seems to me that Cloudformation and Terraform take the declarative approach to cloud management, and there is no tool for the imperative approach. (Cloudformation has Change Sets but you can't write those or edit them; they just tell you its plan.) Resizing your database's EBS volume may be an example where you want to give careful instructions. Try that in Cloudformation and it will destroy the old volume and attach a new one. :-)
I'd love a tool that lets you keep a history of "cloud migrations", where you write both the "up" and the "down". It could be as simple as some organization around a bunch of fog/boto/aws-sdk scripts, although it would be even nicer if it had helper methods like in Rails, e.g. add_autoscaling_group, and those would usually know how to reverse themselves.
The really great feature would be to have "transactional DDL" for cloud migrations. With database migrations you get this for Postgres but not MySQL, and it's really nice. I know this isn't 100% possible, but CloudFormation comes close already with its rollback attempts. You'd definitely need your own layer of helper methods for this, because I can't imagine implementing it without something like a Command pattern. (The way AWS auto-generates their API tools, maybe it wouldn't even be that difficult to generate an identical API that just records what you want to do.)
I've pitched this idea a few times to AWS people, but maybe you should do it instead. :-)
Anyway, Hashicorp has contributed a ton to my developer happiness, so I wish you great success. Oh, and congratulations on your wedding!
I've asked aws in the past for an API that lets you describe a sequence of changes in a batch, it tells you whether there's any problems with the sequence (like "at step nine of this batch, you tried to create an instance in a vpc subnet that was deleted by step four of this batch"). And then if there are no issues with the batch, another API lets you have aws execute the batch. It sounds a lot like the DDL transactions you mentioned.
I'd love a tool that lets you keep a history of "cloud migrations", where you write both the "up" and the "down".
We did a dirty solution for this with Ansible. We have a base playbook (with roles and all that jazz) that sets up a server and then we can write patches which are separate playbooks.
After the patch has run we simply store the patchlevel in a file on the server so that we dont "double-apply" any patches on the next patch-run.
I wrote a tool to auto migrate postres using HashiCorp Config Language. It has the issues you describe — hard to rename tables and adding indexes can take a long time on existing data. It’s pretty convenient if you work within the constraints however.
We're going to be getting deep into multi-cloud (via acquisition), so your thesis on the topic was interesting to read.
I remember reading that original post all those years back and have built with all of your tools, so it'll be a thrill to dig into using Hashicorp as we do this. :)
Can you talk a bit about your password management bits? One of the other programs within our shop said they were going to use (or were doing a pilot) of password management with your kit.
I love Hashicorp products and i personally used most of them throughout my career, so big congratulations. The only thing i was always wondering is how are you planning to make money? It feels a bit like Docker - great technology, supported by big fundings, but the business side of things is very immature...
We're not comfortable talking about revenue publicly, but HashiCorp is an extremely healthy business at the moment. We've doubled revenue ( at least, maybe more ;) ) two years in a row and thats from a starting point of 7-figures. If we chose to halt hiring and growth, we could very easily be profitable today even with the 350 employees we have. But we're still seeing really strong growth and so we're continuing to invest!
Great news! The HashiCorp stack has really made my infrastructure work a much more enjoyable thing. Nomad and Consul in particular are wonderful antidotes to the complexity of Kubernetes for setups that don't require all of its features.
Terraform has become a big part of my professional life. It has become a force multiplier in my ability to solve large problems. I've liked it enough to regularly contribute to provisioners and make documentation improvements.
I agree with the parent comment. Some of the tooling is a bit rough around the edges from a UX perspective, but has been getting better over time. But in terms of software built by developers for developers to solve real developer issues, Hashicorp produces some of the most consistently useful and well built software I’ve ever used.
Tools like Consul and Vault have become a staple in a lot of the companies doing cloud. It’ll be interesting to see what happens as CNCF picks up steam, but it’s been great software nonetheless.
Terraform is the best you can get right now but it’s still so incredibly painful to use. Every team I’ve ever been a part of is ultimately building a custom framework on top of terraform to spit out .tf files. It’s so frustrating.
I wish our HashiConf videos were up but they won't be up for another week or so. But the major theme for our Terraform announcements was "listening & reacting." We've been spending a lot of time with users/customers and listening to their feedback and we wanted to show that we're committed to that. Without going into the whole thing, a couple highlights are Terraform 0.12 and our commitment to collaboration for everyone. Both linked below. Hopefully these help a lot but we're definitely trying to make it easier to use across the board!
All of the changes for 0.12 HCL look great! I love declarative infrastructure but some of the tricks to iterate a map etc were hacks (using count as a boolean etc).
They hired a TerraForm customer named Kristin Laemmert away from Nike and she gave an in depth talk on new language features in HCL such as for loops. She demonstrated a ton of before and after code snippets that included hacks she had to make in HCL to get things to work.
She was pretty clear at least some of the work was in-flight before she started at Hashi so it seems they're using community input as well.
It's really painful but TBH there's nothing I've come across that's much better. AWS' own tools for managing this are abysmally complicated or lack versioning, etc.
We use a git repo to manage our terraform configs and it does the job, albeit in a convoluted fashion.
This is very true, but it is fairly easy (in that you have to figure out the DSL bits) to abstract it out (into a module) so that your resource (not provisioner) is just calling a script to setup the box specific to the cloud provider.
I started with Ansible and quickly realized it was the Apache Ant of devops. I was programming in yaml, which is really convoluted... =(
I moved onto Terraform and in less than a page of fairly simple DSL, I can spin any number of instances up in the cloud across availability zones, attach storage, setup the box, add the IP -> DNS mapping in cloudflare. Part of what makes this work really well is they have the concept of storing state between runs as part of the core framework, which is something Ansible is missing, without using an external service.
It isn't perfect, but I'm super impressed with what Hashicorp has done.
Yeah so tf and cloudformation are kind of the big options in the aws space. They’re both good tools. But I like tf over cf because I have additional control.
The import functionality is super useful in real life, as is “state hacking”. Yeah so technically it’s an antipattern but when you are down and dirty cloud engineering it is very useful to be able to edit the tf state (by comparison you have no access to the internals of cf) to avoid destroying/migrating databases or other precious resources where it turns out you judged the proper point of ownership wrong. This is not really the reason one is “supposed” to like one tool over another but there it is.
Refactoring infrastructure as code is kind of a dark art, and IMHO tf has a way better story for that. SUPER hard to sell though because the problems are not obvious at all. The main reason is that it exposes state as a first-class concept and although big chunks of that are presented as “internals you should not mess around with” it is a pretty big win in terms of day-to-day engineering (and of course usually you shouldn’t...).
What I do wish is that HashiCorp would finally make using third party providers suck less. They seem super sold on a “registry” and bunch of stuff which solves HC level problems while I would be happy with a URL and a trust relationship...
Yeah so tf and cloudformation are kind of the big options in the aws space. They’re both good tools. But I like tf over cf because I have additional control.
I love Consul+Nomad for job scheduling for on prem deployments. Especially since you can use regular executables and not just Docker containers.
But for AWS, it’s CloudFormation all of the way. Not necessarily because it’s better, but if something weird happens that I can’t figure out, I have an easy button - the business support level of AWS.
Except the fact that multi cloud doesnt work and I don't think it will. It adds too much complexity also costs are the big issue too since clouds price data transfer out quite heavily
Consul, Nomad, Terraform, and Vault are the only products that have associated enterprise products (in addition to being open source). Different products have different impacts different quarters but all of them are contributing many millions per year individually.
Vagrant and Packer do not have enterprise products and we don't try to monetize them much (Vagrant Cloud has some paid plans for box hosting but primarily so that we remain cost neutral). However, we're fully committed to them and both teams have full time engineering and management assigned to them. They're not deprecated in any way.
With as useful as both Vagrant and Packer are to users... I don't understand how they don't contribute to the bottom line equally, or at least in a significant way.
For any OSS products that claims to make money you can check their product page and see if they put out some pricing information. For example https://www.hashicorp.com/products/consul
The project that really kicked off HashiCorp was Vagrant, and I "launched" it right here on HN over 8 years ago: https://news.ycombinator.com/item?id=1175901 So its always fun to see things come back around. :) For those that are less familiar these days, HashiCorp now has many major OSS projects: Vagrant, Packer, Consul, Terraform, Vault, Nomad (in addition to lots of minor ones and libraries) addressing different areas.
For the HN crowd, there are a couple background viewpoints that I would also like to share that may be helpful on painting the backdrop on the product and business behind HashiCorp. A first quick one is my tweet on the four dimensions for "multi-cloud": https://twitter.com/mitchellh/status/1022162653618135040 At HashiCorp we're mostly looking at enabling #1 (workflow portability), though that has downstream effects of helping with the other dimensions as well.
Second, I often get asked "Why multi-cloud?" And I wrote up a fairly long answer on Reddit about that with real examples that I recommend: https://www.reddit.com/r/devops/comments/91afzz/why_multiclo...
While I'm happy to answer any questions, this post is rather late for me local time (past 10:30 PM here) and I'm leaving tomorrow to get married this weekend so I can't promise anything!