"Definitely been on the other side of the table arguing for a quick 3-hour script."
But that's probably the time it took to write it, right? 80% of the cost of software is in maintenance, so there's another 12 hours worth of maintenance left to account for. If you know you're going to spend 15 hours on it, then you might as well use a system you know will cost less to extend or scale over time.
"We've been very careful to minimise Airbyte's learning curve."
That's good for quickly onboarding new customers, but not necessarily for the system to be scalable or extensible.
"Starting up Airbyte is as easy as checking out the git repo and running 'docker compose up'."
I'm always curious about this. Docker-compose doesn't run on more than a single host, unless you're using it with the AWS ECS integration (and maybe Swarm?). So sure, the developer can "get it running" quickly to look at it, but to actually deploy something to production they'll have to rewrite the docker-compose thing in something else. If you provide them with Terraform modules or a Helm chart, that would get them into production faster. And maybe even a canned CI/CD pipeline in a container so they can start iterating on it immediately. It's more work for your company, but it shortens the friction for the developers to get to production, and enables businesses to start using your product in production immediately, which I think is a pretty big differentiator of business value.
"If you know you're going to spend 15 hours on it, then you might as well use a system you know will cost less to extend or scale over time."
I wish younger me realised that earlier :)
"And maybe even a canned CI/CD pipeline in a container so they can start iterating on it immediately."
Definitely! Although a good number of users are surprisingly happy with their Airbyte instances on a single node.
We do have a Kubernetes offering for those looking to scale Airbyte beyond a single node. We also have Kustomise/Helm deploys for this, though I'll be the first to admit that the Helm charts are mostly community-maintained and can be improved. This is one of our (my) top priorities going into the next Quarter.
But that's probably the time it took to write it, right? 80% of the cost of software is in maintenance, so there's another 12 hours worth of maintenance left to account for. If you know you're going to spend 15 hours on it, then you might as well use a system you know will cost less to extend or scale over time.
"We've been very careful to minimise Airbyte's learning curve."
That's good for quickly onboarding new customers, but not necessarily for the system to be scalable or extensible.
"Starting up Airbyte is as easy as checking out the git repo and running 'docker compose up'."
I'm always curious about this. Docker-compose doesn't run on more than a single host, unless you're using it with the AWS ECS integration (and maybe Swarm?). So sure, the developer can "get it running" quickly to look at it, but to actually deploy something to production they'll have to rewrite the docker-compose thing in something else. If you provide them with Terraform modules or a Helm chart, that would get them into production faster. And maybe even a canned CI/CD pipeline in a container so they can start iterating on it immediately. It's more work for your company, but it shortens the friction for the developers to get to production, and enables businesses to start using your product in production immediately, which I think is a pretty big differentiator of business value.