> Running your own datacenter in 2023 is incredibly risky.
There are middle grounds.
But let's be honest: 99% of companies have never done the napkin math, because nobody ever got fired for choosing IBM^W AWS.
We joked about this in my company: we had a variable-load thing that we used autoscaling in the cloud for, but it had a baseline load that purchasing a real machine might have made a lot of sense for. The napkin math probably checked out. We never suggested it more than jokingly, though, because even when we suggested it jokingly, we got shut down: "You don't understand the cost of that." No, actually, we jokingly did enough math that we do understand, better than the people criticizing us did. We never did it.
Whenever the "own it" argument comes up, eveybody is real quick to hop on the "but maintenance cost" train. But as I perceive it, those who believe in the cloud budget exactly $0 for maintenance of managed cloud resources. As someone who's only done cloud, that number is unadulterated bullshit: the number of hours I've had to spend chasing cloud vendors to do the job that we're paying for is just silently flying under the budget radar. In the minds of the finance books, I'm 100% SWE, but in reality, I'm 75% SWE, and 25% support ticket monkey.
At least with a real machine, it'd be interesting, and I'd have some agency to actually solve the problem. As a support ticket monkey, I'm utterly powerless. I'm tired of having to beg.
That's not to say I'd move everything off cloud; I actually think the vast majority of what we do is well-suited for cloud, mostly because upper management can't make up their mind about product direction enough to be able to say "yes, we can purchase this and we'll use it." But those nuggets of stability do happen from time to time.
> disaster planning
"Disaster planning" is something every org wants, because they're trying to tick the box with the regulator. But the requirements that get passed down border on absurd: "what if a meteor hit AWS and they were never able to recover from it?" … we're literally never going to plan for that, because the $ needed for that level of eng. work is not going to happen. A sane scenario would be "can we handle an AZ outage?" (or, let's start there, and maybe, maybe if we can get that down pat, then we can graduate to regional outages.)
> cyberinsurance
… you don't get out of this via being in the cloud, if you need it. (I wish we did, because ours pushes some utter inane requirements.) I can mismanage a machine in a DC just as easily as I can mismanage a VM in the cloud.
> Organizations pay attention to dollars
No they don't. This oft-repeated mantra is nonsense. Finance dept. get an invoice that has a total; even were they to have access to the finer billing information, they're not technical, and cannot understand it. I've yet to be at a company that's dedicated sufficient resources towards infra eng such that we could do the legwork necessary to present a sane organizational view of what cloud infra dollars go to what high-level objectives or teams. The resource tagging isn't there, and even if it were, some things cannot be tagged, and you still have to aggregate bills from a dozen different vendors, and then figure out what weights to apply to shared resources across OUs. I'm on employer #4? and have yet to see anyone scratch the surface of that.
Which is why you see articles about cloud $ waste all the time.
What happens far more often in my life is someone from management descending with "why are we spending $X on Y?", where $X is usually an order of magnitude wrong, or Y is … something we're not even doing anymore? And then you have to go round the mulberry bush of "how did you arrive at that figure?" "okay so here's what those numbers mean" "here you're adding $/mo and $/yr and you can't do that"
> Do you really think other (smaller) orgs can do a better job at hosting a datacenter than Amazon / Google / Microsoft / Cloudflare?
Than Microsoft? Absolutely yes. The others, probably not.
> Yes, I get it. All the computer processing power in a handful of actor's hands is probably not the most fantastic thing.
The long-term end state of not investing money into R&D is that it is centralized into those who do, and you become beholden to them. You get what you pay for, here. It's not good, and I think there's discussion to be had around that, but my real problem is the cognitive dissonance that follows. If you want to centralize on one of the cloud duopoly, then you also need to acknowledge that your own eng cannot be held responsible for the cloud's reliability: they have no control over it.
Excellent response, thank you. To expand on the cybersecurity aspect - think of services like Cloudflare WAF and DDoS protection. These services are very easy for orgs to implement and do a really good job at covering 95% of threats quickly.
Could you imagine a 1,000 person org with a 20 person IT department rolling their own DDoS solution?
But yes you are also right, cyber insurance is still required, and even AWS touts an expectation of a “shared responsibility” model.
I’m still skeptical that cloud hosted offerings are a bad thing. For a long time there were only Ford, Chrysler and Chevy in America, then foreign imports became popular, then a few years ago Tesla became a contender.
I still think new entrants can come into the cloud space, particularly in Europe, but they need to do their due diligence and understand their competitors offerings very well.
There are middle grounds.
But let's be honest: 99% of companies have never done the napkin math, because nobody ever got fired for choosing IBM^W AWS.
We joked about this in my company: we had a variable-load thing that we used autoscaling in the cloud for, but it had a baseline load that purchasing a real machine might have made a lot of sense for. The napkin math probably checked out. We never suggested it more than jokingly, though, because even when we suggested it jokingly, we got shut down: "You don't understand the cost of that." No, actually, we jokingly did enough math that we do understand, better than the people criticizing us did. We never did it.
Whenever the "own it" argument comes up, eveybody is real quick to hop on the "but maintenance cost" train. But as I perceive it, those who believe in the cloud budget exactly $0 for maintenance of managed cloud resources. As someone who's only done cloud, that number is unadulterated bullshit: the number of hours I've had to spend chasing cloud vendors to do the job that we're paying for is just silently flying under the budget radar. In the minds of the finance books, I'm 100% SWE, but in reality, I'm 75% SWE, and 25% support ticket monkey.
At least with a real machine, it'd be interesting, and I'd have some agency to actually solve the problem. As a support ticket monkey, I'm utterly powerless. I'm tired of having to beg.
That's not to say I'd move everything off cloud; I actually think the vast majority of what we do is well-suited for cloud, mostly because upper management can't make up their mind about product direction enough to be able to say "yes, we can purchase this and we'll use it." But those nuggets of stability do happen from time to time.
> disaster planning
"Disaster planning" is something every org wants, because they're trying to tick the box with the regulator. But the requirements that get passed down border on absurd: "what if a meteor hit AWS and they were never able to recover from it?" … we're literally never going to plan for that, because the $ needed for that level of eng. work is not going to happen. A sane scenario would be "can we handle an AZ outage?" (or, let's start there, and maybe, maybe if we can get that down pat, then we can graduate to regional outages.)
> cyberinsurance
… you don't get out of this via being in the cloud, if you need it. (I wish we did, because ours pushes some utter inane requirements.) I can mismanage a machine in a DC just as easily as I can mismanage a VM in the cloud.
> Organizations pay attention to dollars
No they don't. This oft-repeated mantra is nonsense. Finance dept. get an invoice that has a total; even were they to have access to the finer billing information, they're not technical, and cannot understand it. I've yet to be at a company that's dedicated sufficient resources towards infra eng such that we could do the legwork necessary to present a sane organizational view of what cloud infra dollars go to what high-level objectives or teams. The resource tagging isn't there, and even if it were, some things cannot be tagged, and you still have to aggregate bills from a dozen different vendors, and then figure out what weights to apply to shared resources across OUs. I'm on employer #4? and have yet to see anyone scratch the surface of that.
Which is why you see articles about cloud $ waste all the time.
What happens far more often in my life is someone from management descending with "why are we spending $X on Y?", where $X is usually an order of magnitude wrong, or Y is … something we're not even doing anymore? And then you have to go round the mulberry bush of "how did you arrive at that figure?" "okay so here's what those numbers mean" "here you're adding $/mo and $/yr and you can't do that"
> Do you really think other (smaller) orgs can do a better job at hosting a datacenter than Amazon / Google / Microsoft / Cloudflare?
Than Microsoft? Absolutely yes. The others, probably not.
> Yes, I get it. All the computer processing power in a handful of actor's hands is probably not the most fantastic thing.
The long-term end state of not investing money into R&D is that it is centralized into those who do, and you become beholden to them. You get what you pay for, here. It's not good, and I think there's discussion to be had around that, but my real problem is the cognitive dissonance that follows. If you want to centralize on one of the cloud duopoly, then you also need to acknowledge that your own eng cannot be held responsible for the cloud's reliability: they have no control over it.