Idk, seems like that would be a pretty standard abstraction layer to implement at a large company.

Sorry if this is an naive question. If every large companies are doing it, why aren't there a standard open source abstraction library available?

I used Jclouds [1] a few years back to do this. While it's very powerful, it's also fairly complex (abstracts not just storage but also compute/etc). At the time, I remember wanting a simpler abstraction layer over it, and ended up building my own. Maybe something like that exists now, I'm not sure.

The devil is in the details of how each provider offers vendor-specific things and you want to take advantage of them. For example Reduced-Redundancy storage is iirc an Amazon-specific offering, or if others offer it, it's probably under different SLA terms/measurements. This rapidly breaks many generic abstractions; maybe this is why everyone ends up writing their own little shim layer for their situation.

In some sense it reminds me a bit of building database-connection-pools in the 90's, before they were really standardized everyone rolled their own and learned all the awful lessons about reference counting along the way. Then along came ODBC, then JDBC, and things were so much easier because you only had to deal with one API, and the databases would conform to their side of it. So I think, isn't that what OpenStack (or something?) is supposed to be for cloud services? But whoa, the depth and complexity of these services far exceeds that of a 90's database. It will take a while -- but over time and with patterns of common use well established, a stable base of standard APIs will abstract away most differences, making things so much nicer. I can dream.

[1] https://jclouds.apache.org/

The S3 API signature seems to be the defacto abstraction. Network appliance supports it for their (on premisis) hardware. Minio supports it in their open source object storage software. The API signature seems to support competition via tags[1] and other features. Not feeling a lot of lock-in specific to S3.

[1] https://docs.aws.amazon.com/AmazonS3/latest/dev/object-taggi...

"Sorry if this is an naive question. If every large companies are doing it, why aren't there a standard open source abstraction library available?"

I think Tahoe LAFS is the open source solution for this.

https://tahoe-lafs.org/trac/tahoe-lafs

https://en.wikipedia.org/wiki/Tahoe-LAFS

There are probably 10 already in existence, but they don't meet some exact legal or technical requirements that these companies demand.

Often the reason to open source is to get developer mindshare (tensorflow) or in the hope of gaining traction to compete against existing closed source tech (opencompute).

Yeah I don't know how many companies do it, but it seems like a process that in some form is a good best practice.

The problem there being that small companies get fooled into thinking they need it and spend a bunch of time abstracting away (and complicating!) a service layer they'll likely never change.

Yeah, as if you could whip this up in an afternoon.

I didn't realize that was a requirement for something being standard...

Every fortune 50 company has a similar system at this point; you tie it in with your enterprise architecture platform so you can go directly from design to infrastructure using tested patterns.

Pretty much table stakes in large enterprise at this point; and as usual IBM / SAP / Oracle / SoftwareAG cabal have bought up everything that matters.

There used to be a project called Geneva. Not sure if this is same.