Unpopular opinion...but imo open source has, completely by accident, caused the centralization of the internet. Because software licensing is less profitable, we've since gotten all the big software makers into the server game, and their infrastructure becomes what you are licensing instead. Their infrastructure could be based on something OpenSource, but you might still be vendor locked into AWS, Azure, GCS, etc.
Turned out to be a very popular opinion [1] that OSS having commoditized the cost to develop Software to near zero that the primary benefactors are the major cloud hosting providers who are able to extract rent from hosting OSS software. Which are using their consolidation of wealth to build out their billion dollar infrastructure moats ensuring barriers of entry that no-one else will be able to participate in.
The richer and more powerful the Cloud Hosting Oligopolies get, the worse off it will be for Indie OSS developers who will be unable to compete with their network lock-in, paid resources, marketeers + advocates, advertising, etc.
What's worse is if you release OSS software the Cloud Vendor network monopolies naturally stand to make more revenue then any attempts in trying to host it yourself, making it impossible to compete against given the more effort you put into development, the more revenue you will create for them which they in-turn can use to fund their competing efforts.
Personally I'm all for companies like Redis Labs, Elastic, Confluent and MongoDB who realize the threat and start distributing their future investments under "OSS free that's free for everyone else except major cloud vendors" to force the cloud oligopolies to revenue share back some of the money they've made from hosting their software. Hopefully we'll see more of OSS follow which will force them to implement a system for revenue sharing back to OSS software they're getting paid to host.
Even when they're unable to license it they still have the luxury to sit back and wait to see what software becomes popular and proves itself in the market before instructing their army of paid devs to clone it as they've done with their MongoDB clones that implement the MongoDB protocol (also requires a lot more effort to create something with constant customer feedback + iteration, then it is to clone it). Thanks to the integration with their ecosystem ensures their investments will have better ROI and popularity then the original authors could ever have.
I agree, even wrote a quaint blog post on that long ago https://gondwanaland.com/mlog/2006/08/07/aolternative-histor... ... but partiallypro seemed to be claiming the close to the opposite, that open source caused centralization. I'd still love to read a justification for that claim!
there is no internet there is only the bitcoinstream.
The early internet was "opensource like" as in not monetized.
it was a very different time in the 80s and early 90s.
Grassroots BBs and FTPs were the fodder.
websites were made to convey information, many of them made for the love of it.
i think no FOSS would have allowed the current state of the internet to occur much more rapidly.
many more people would have experienced the net/web as a service, extended to you on the terms of the service provider.
All of the internet was built on FOSS. If you want to understand what the world would have been like without FOSS, you need to look not at the internet, but at the competing technologies that were duking it out at the time. Novell Netware for the bare backbone and Compuserve/AOL for services. Keep in mind that these services charged per hour for usage. Even when we talk about grass roots systems like BBSes (the vast majority of which were FOSS, BTW), the task to connect them up together was ridiculously expensive. I remember paying $4.50 per hour plus 1 cent per bit transferred to interconnect my BBS. It's one of the reasons I switched to Usenet and UUCP email as soon as I knew what it was.
Back then there were gatekeepers who made sure you paid to play. The internet broke that because it allowed anybody to connect for free as long as you found a way to do it. The gatekeepers used proprietary protocols and software to lock you into their paywall. The internet used FOSS to make it easy for you to connect. That was literally the difference.
And for whatever reason everyone bought the rhetoric that you could save so much money by moving your stuff to these services in the "cloud" though there is no clear reason to think that this is true. You still need employees to manage those services, you still need to backup stuff and restore it if that stuff fails. etc.
If you're doing a "lift and shift" then running stuff in the cloud is almost certainly going to be more expensive.
However if you migrate to using managed services then there can be a lot of cost savings.
To be honest it's a lot like the old days of PHP hosts - comparing VPS to shared hosting - except it's now the shared hosting is scaled up and "enterprisey".
What I do find most people overlook when saying "the cloud is cheap" is just how much time is required to deploy to the cloud. I don't mean POCing a few things in the AWS console but rather deploying stuff properly using Terraform, Puppet, etc and architecting it correctly so you have resilience without paying through the nose for it.
Because you buy these services, you can blame someone else when they fail. That's why cloud services are favored among IT staff. You can no longer get fired for the backup failing if it's Google Compute that failed to back up your instances.
Ha. Yeah right, that assumes the boss is stupid enough not to know who recommended using Google Compute and what justifications they used. Don't worry, they'll be asking all sorts of questions while you're twiddling your thumbs, unable to do anything at all about the infrastructure being unavailable.
That's certainly an attractive aspect of the cloud but in my experience what tends to be the bigger driving factor is because you're effectively paying for your hardware on a subscription basis rather than paying tens or hundreds of thousands of pounds / dollars up front.
If your backups fail then you still get fired, but now it's for choosing an unreliable cloud vendor.
If you don't want to pay for your hardware up front, most hardware vendors are happy to sell it on an installment plan.
I think that "cloud" started taking off right after x64 servers got hardware virtualization support, and then you had cloud vendors showing IT departments how much more cost effective their virtual machines were than the single-application physical machines that were the status quo ante. Not bothering to mention that they're frequently not more cost effective than hosting the virtual machines yourself.
> If your backups fail then you still get fired, but now it's for choosing an unreliable cloud vendor
It wasn't me that came up with the backup example - which I do think is the best example of the point they were making because you wouldn't expect that particular service to fail nor someone to get fired. And if it does, it's more likely yourselves to blame for setting it up wrong than it is the cloud vendor at fault.
Their general point was more that if you self-host and the infra goes down due to hardware failure (eg switch goes pop) then it's up to you to fix. With cloud services that responsibility now becomes someone else's problem.
Personally I find this the least convincing reason to move to the cloud because I'd rather be proactive fixing something than waiting for someone else. However it's undeniable that it is an incentive to some - I suspect more for manager types than engineers though.
> If you don't want to pay for your hardware up front, most hardware vendors are happy to sell it on an installment plan.
You pay a premium for that (interest and insurance) and are still stuck with the repayments even if your project crash and burns before the instalment plan finishes. If you're a business and looking to buy hardware you are much better off buying it outright - you have a stronger position to negotiate a discount (so you're not paying advertised price) and you have capital you can resell if the worst happens. However if your a small or medium sized company and you don't have any self-hosted equipment currently or are looking to re-outfit several racks of gear, then the cloud is a very attractive offering.
> I think that "cloud" started taking off right after x64 servers got hardware virtualization support, and then you had cloud vendors showing IT departments how much more cost effective their virtual machines were than the single-application physical machines that were the status quo ante. Not bothering to mention that they're frequently not more cost effective than hosting the virtual machines yourself.
It sounds like your impression of the cloud is massively outdated. If you're just looking for a VMWare equivalent then AWS et al isn't going to be that impressive. Where public clouds have overtaken self-hosted solutions is with their SaaS solutions. This is why in my other post I make the distinction between "lift and shift" type deployments on the cloud, and using their SaaS offerings.
I do totally get the appeal of self hosting though. I've spent the majority of my career in server rooms and love the feeling of owning my own equipment. However pragmatically I can also see why the cloud is such an attractive solution to businesses these days.
> It wasn't me that came up with the backup example - which I do think is the best example of the point they were making because you wouldn't expect that particular service to fail nor someone to get fired. And if it does, it's more likely yourselves to blame for setting it up wrong than it is the cloud vendor at fault.
Backups are notorious for going untested because too often nobody cares if they work until you discover the hard way that they don't. That's the "setting it up wrong" problem and cloud doesn't really change that.
Cloud vendors then add a different problem, which is that they operate at a scale that brings multiple redundancy failures into the realm of plausibility. Then, because the same infrastructure is shared by many customers, you end up with greater exposure to those kinds of previously negligible probability multiple systems failures when something happens that affects thousands of customers at once.
Which is still not very common, but it can be very bad when it does.
> Their general point was more that if you self-host and the infra goes down due to hardware failure (eg switch goes pop) then it's up to you to fix. With cloud services that responsibility now becomes someone else's problem.
There are two components to this. One is whether it's "your problem" in a CYA sense, which I understood to be the original argument, i.e. if the system fails and your company loses ten million dollars then you lose your job. But that hasn't really changed. If the backups don't work, pointing at the vendor is not very effective cover when you're the one who chose them.
The point you're making is that when something fails you don't have to spend time fixing it because that's somebody else's job now. Which is fine, but then you're paying a premium for that and the question becomes whether such hardware failures are common enough that you come out ahead that way or not.
> You pay a premium for that (interest and insurance) and are still stuck with the repayments even if your project crash and burns before the instalment plan finishes.
You pay even more of a premium for third party hosting, and server hardware is a general purpose commodity. If your project ends then it can be used for the next one. And even something bought on credit can be resold and the money used to pay off most/all of the debt, especially now that Moore's Law is in decline and hardware doesn't depreciate as fast as it used to.
Moreover, if you know ahead of time that your project has a high short-term failure probability then it can make sense to not want a longer-term commitment, but not everything is so uncertain. Many projects are known to have a >95% probability of still existing in five years.
> If you're a business and looking to buy hardware you are much better off buying it outright - you have a stronger position to negotiate a discount (so you're not paying advertised price) and you have capital you can resell if the worst happens.
That's true. But what if for some reason you prefer "effectively paying for your hardware on a subscription basis rather than paying tens or hundreds of thousands of pounds / dollars up front"? :)
> If you're just looking for a VMWare equivalent then AWS et al isn't going to be that impressive. Where public clouds have overtaken self-hosted solutions is with their SaaS solutions. This is why in my other post I make the distinction between "lift and shift" type deployments on the cloud, and using their SaaS offerings.
I don't think we're really disagreeing all that much. There are circumstances where cloud hosting makes sense, particularly when your needs vary unpredictably over time.
My point is that people overuse it. You see companies with entirely static and predictable future needs moving their entire operations into the cloud anyway.
And I suspect people are going to regret the long-term experience of SaaS. Once you have a decade of your business data on someone else's proprietary service with no export function, what does your negotiating position look like?
> Backups are notorious for going untested because too often nobody cares if they work until you discover the hard way that they don't. That's the "setting it up wrong" problem and cloud doesn't really change that.
At risk of repeating myself:
1/ I didn't make the backup comparison
2/ I've already said I didn't agree with GP using that specific example
3/ I didn't make the backup comparison
So can we drop the discussion about backups please.
> The point you're making is that when something fails you don't have to spend time fixing it because that's somebody else's job now. Which is fine, but then you're paying a premium for that and the question becomes whether such hardware failures are common enough that you come out ahead that way or not.
You can argue this all you like however that doesn't change the fact that I'm only reporting what I've observed some decision makers base their decisions on. You may not agree with their opinion but arguing with me isn't going to change that.
> You pay even more of a premium for third party hosting, and server hardware is a general purpose commodity. If your project ends then it can be used for the next one. And even something bought on credit can be resold and the money used to pay off most/all of the debt, especially now that Moore's Law is in decline and hardware doesn't depreciate as fast as it used to.
As I've already said several times, it depends on how you utilise the cloud. Lift and shift projects where you replicate an on-prem solution in the cloud often works out more expensive. However using SaaS solutions can work out cheaper - if you know what you're doing. It's really not a clear cut as you think.
> Moreover, if you know ahead of time that your project has a high short-term failure probability then it can make sense to not want a longer-term commitment, but not everything is so uncertain. Many projects are known to have a >95% probability of still existing in five years.
If it were as easy as that then significantly fewer startups would fail. I get you don't like cloud services but your exaggerations are now bordering on the ridiculous.
> That's true. But what if for some reason you prefer "effectively paying for your hardware on a subscription basis rather than paying tens or hundreds of thousands of pounds / dollars up front"? :)
If that's what you want to do then do it. If you want to store all of your hardware inside a rocket and shoot it into space then you're free to do that too. It doesn't mean any of them are sensible suggestions though. But if that's what you prefer to do then go for it ;)
> My point is that people overuse it. You see companies with entirely static and predictable future needs moving their entire operations into the cloud anyway.
> And I suspect people are going to regret the long-term experience of SaaS. Once you have a decade of your business data on someone else's proprietary service with no export function, what does your negotiating position look like?
Of course people overuse it. That's how IT industry works. Someone comes up with a good idea and then everyone rushes to clone it regardless of whether it makes sense or not. As much as we'd like to consider ourselves intelligent engineers our industry is still full of hypes and fads. Often it's cyclic too - with new generations of engineers re-discovering old technology and rebranding it. I've seen it happen time and time again in my many years in IT.
I'd like to add...no one (read: users / consumers) care. It could be Martian source and as long as it worked 99.9% of people would be fine with it.
The problem is, OSS has changed the biz model for software development, but we're still stuck with bugs, poor design, shite UX, frustrations, etc.
Yes, OSS solved a problem. But it wasn't The Problem. So now what?
And while I do embrace the OSS ideal, I wish I could total up the hours lost to vague documentation, delays in support (if support even exists), etc. For example, you can probably build a damn good modern CMS with all the time lost to the WordPress codex.
The cloud SaaS is basically a buisness model. The user cant just download the software and host it him/herself. When the user dont pay you can just cut off access.
So in the buisness angle its briliant. From a tecnical standpoint its stupid as Personal computers today is very capable to run most cloud software itself. And we have reliable networks for when networking is required. Most PC's are online all the time, just like servers, and have dual core processors, just like the servers. I wish there where better business models for free open source software.
You’re free to host stuff on a desktop PC if you want. Some people do. However the devil is in the details. For example you was a redundant power supply, then you’ll need a server case with dual PSUs, two power rails backed by two UPSs. You’d probably then want some kind of out-of-band management console which means you then need a VPN server too. And of course that would need the same redundancy with power too. And all that is fine until you realise your router is a central point of failure, so you’d then want two switches and two hardware firewalls. But what if the hardware failes on your PC? You’d then want a battery-backed RAID controller and a device for making off site backups (since you’re not using the cloud, that probably means a tape drive or DVD writer- depending on the volume of data you’re storing). However you’re still reliant on one PC so you’d need to double that up too. And since you now have 4 network devices, 4 servers and 2 UPSs, you’d probably need to store than investment somewhere secure - like a dedicated room with a camera and keypad lock on the door. Best install some air conditioning too while you’re at it. And you’ve still yet to address the looming problem of the builders next door accidentally digging through your leased line (this has literally happened to me before).
Costs can quickly spiral when you’re self-hosting internet services for paying customers who expect their internet services available 24/7. Which is why the cloud is so attractive - it allows you to build your product without paying the upfront cost of self hosting.
That all said, I do honestly get the appeal of self hosting. It’s fun owning your own hardware. That is until it fails....
You got resilience pretty wrong here. If you start with a single PC at home and a single ISP, you actually get like three nines of availability, not everywhere of course, but in urban areas you mostly do. Which is almost exactly what a datacenter can get you. But interestingly enough, you can get even better resilience at home than in a datacenter by having just another PC connected to another independent ISP. That's because an ISP is a pretty significant point of failure and you can't get another independent ISP in a datacenter (not that you need it, as using a different independent datacenter instead is much better for resilience).
ok, let’s play out your scenario: so now you need to buy two houses in two different towns so you get two different ISPs and running your server on two different power substations (since you’re no longer building that resilience in at the server room level). That is still going to cost you way way way more than deploying something on AWS in a multi a-z or multi region. And you still haven’t addressed the issue of secure out-of-band management.
The only benefit with your solution is that the second home is at least an capital asset; but even still, it’s more than any small or medium sized company would be willing to stump up (I say this from experience since I’ve worked on disaster recovery solutions for SMBs in the pre-cloud computing era)
I guess you didn't get my point. You can subscribe to multiple independent ISPs in one house. You don't get that while renting a server in a datacenter.
I did get your point, however you’re still dependent on one pipe into the house which then doesn’t solve the issue I highlighted. You could buy additional leased lines with a different physical route out but those are not cheap either and you still haven’t solved the issue of redundancy in any other system aside your gateway.
The complexity grows the more users you need to serve. And the available percentage you need. For example, maintaining your own e-mail client, on your own computer, is not that much work, compared to what Google needs to host Gmail.com
Absolutely agree. I do actually run a server in my house for personal services however the difference between running personal infrastructure and running a service for paying customers is night and day. You may not need your own data centres ala Google but that doesn’t mean the only other option is running everything on a PC stashed away in your closet.
In fact the very advantage of cloud computing is you can scale your infrastructure to meet your demand (and I don’t just mean auto-scaling; but also the instance sizes you run and infrastructure you deploy). Which is the point I was making in my previous post: you can build the infrastructure which suits your problem and let the cloud provider deal with many of the “enterprisey” problems for you. Then when you reach the point where you need to start worrying about the big problems, well you can decide how you want to design your solutions there after.
Does cloud infrastructure have to be centralized for the foreseeable future? I'd imagine there's a bunch of people even on HN working on ways to decentralize compute, something like you run a shell command and part of your computers resources are now allocated for anyone part of that ecosystem to use.
I'm keeping a close eye on the Golem network (https://golem.network/). However, they mostly concentrate on distributing heavy-weight computing like rendering or ML, not website hosting.
