I've been working almost exclusively in the AWS space for about 10 years now. Clients anywhere from tiny little three-person consultancies to Fortune 100. Commercial, govcloud, dozens of clients.
Never once have I ever found a use case for making public EBS snapshots.
Who on Earth is thinking that it is a good idea to take an EBS snapshot and make it public?
Note, several of those engagements did involve multiple accounts, and the need to share / copy AMIs and/or snapshots between accounts. But never making them public.
This 100%. I don't do much AWS stuff - but this issue exists outside of AWS as well (inside a corp).
It's why I hate all the password rotation, double VPN stuff - people work around it too much.
All the old staff start having the tech person track their passwords or write them near the computer, all the young staff use whatever new .io domain does X super easily (but less securely) or stick things on thumb driver (no keypad) or use their personal google drive etc.
The later in the day, the bigger the rationalizations.
At some point you are too stupid to be allowed to use a keyboard. And yet we have an entire culture around staying late to get stuff done that probably could have waited until tomorrow.
As much as I dislike the typical suit wearer, the industry as a whole would be much better if IT-people wouldn’t blame everything on them and just tell them you won’t do it. If you are a welder and you boss wants you to weld a gas tank that hasn’t been emptied you tell him that he has no idea what the thing he asks for means, explain why and wait till the gas tank is emptied and everything is checked.
I worked as a Camera guy in film and I am known to be very fast – yet I had directors who wanted things even faster. Bit there is a natural limit to how fast you can get something done without having worthless garbage as a result.
You can take certain risks, skip certain advisable steps, focus on the most essential thing etc, but below that there waits literally nothing.
In my experience taking a step back, breathing in, out, and then proceed to doing it properly is in most cases faster than following your boss into panic and ditching common sense.
It is your responsibility as a professional to say “No” or “Stop” in certain circumstances. And if they really want you to do it, write down the possible consequences of what they force you to do and make them sign that they take the full responsibility.
There is so much talk about IT security with people frowning upon silly behaviour, yet any other craft would bend over backwards before you could force them into unsafe behaviour. If engineers would build bridges like we in IT operate, we would have many collapses a day.
If you want to be more professional about this stuff, build up a Fuck Off Fund. Women in particular have written about fuck off funds in the context of making sure you don't have to nod along when HR says the VP who stuffed his hand down your top was "just fooling around" but - most people need that financial security any time they have to confront the boss. Save to be able to look the big boss in the face and tell them "Fuck Off". "Fuck Off" isn't the response you need when they tell you they want the database authentication disabled "just until Monday, Tuesday at the latest" that's when you want "No". But you need to know you _can_ tell them to "Fuck Off" so you actually say "No". Otherwise you may find yourself agreeing anyway.
For the youngsters out there, don't elaborate when you say "No". That is, don't mention your FOF. That weakens your position.
It is sufficient to look the manager/executive threatening you square in the eye, and state your position with deliberateness. Keep it professional, no raised voices, and be willing to walk away without hesitation if the other side gets abusive.
The really bad ones are those who tell you that if you step through that door don't bother coming back or similar, so be absolutely ready to commit. If you do the main event behind closed doors one-on-one and get that threat as you walk out, sometimes they'll come back with sugar suggesting a do-over. Generally Admiral Ackbar is right in this context, but it's your call.
The negotiation leverage that comes from the FOF is most powerfully communicated non-verbally and in a face-to-face setting, also through body language. The difference is very noticeable between those who have an FOF and those who don't, if you have enough experience. It can be faked, but it takes exceptional practice to fake. The tell starts with how fast and confident the "No" comes back.
This is the nuclear option of course. Exhaust all other avenues of reasonable negotiation first. Like an email with witnesses you pick for deliberately violating departmental policy, for example.
All of these anecdotes have something in common. That the person who is asked to do something is informed about proper procedures. That is uncommon in IT, and especially in companies where proper procedure is too costly and not worth pursuing.
It is still a communications problem. They want you to do the minimum. If I as a camera person do the real minimum the director will be pleased on set but they will get a heart attack in the editing room.
It easier to give in, take short cuts and produce garbage than explaining them under extreme stress that everything faster will result in nothing.
In my eyes it is comparable and I worked in both industries. The difference is, that as a DOP in film your name will be associated with the mess on the screen, while in IT the link is not that clear. This makes the difference.
And unless your director has a monitor (which they sometimes don’t) they cannot tell at all what you are doing. They wouldn’t even realize if you didn’t even switch the camera on.
Oh no. I work with very trendy Silicon Valley folks and it’s always someone in shorts and a t-shirt. Which is to say: it’s definitely a culture issue, but it doesn’t have to be a PHB doing it. I know a lot of really smart people who are under a lot of pressure, and combined with a lack of AWS knowledge, “just make it public” is surprisingly common.
He must have been doing a better job of shielding us from his bosses than I thought, or I've had more crazy bosses than I thought, because a lot of it seemed like normal bureaucratic pathology to me, if a little more intense than usual.
Are you saying there's some sort of group or club where you pay 15 cents for a mask and then put it on to get well-acquainted to psycologically aberrant individuals from a comfortable distance before they go prison?
Fifteen cents is roughly the cost of a small informational pamphlet (likely more if printed with color and laminated but I have taken some artistic liberty in my previous comment), and prison is just one solution to a wider class of social impairments of which sociopathy is just one instance. Other instances of solutions share the quality of being unpleasant but necessary things any civilized society devotes energy to by learned necessity and include a functioning constitution, military, legislature, courts, police, and more. I thought this was clear but on rereading I see maybe I have failed; thanks for asking.
Aren’t public EBS snapshots the underlying mechanism for public AMIs? I’ve ran into complex permissions in a golden image deploy model where the same AMI is used across multiple accounts.
There needs to be controls like S3 where you can explicitly block public data.
>Who on Earth is thinking that it is a good idea to take an EBS snapshot and make it public?
Maybe they're trying to reproduce functionality of docker? It would actually be extremely useful for research involving modeling/AI because you could trivially reproduce the results by bundling the exact code and data.
Edit: actually maybe I'm confusing EBS snapshots with AMIs...
Public EBS snapshots are great, and thankfully a design other clouds didn't copy. I've found all kinds of stuff in there, including a 900GB Oracle backup of a publicly traded manufacturer's accounting system. It doesn't require much imagination to understand how this kind of data could be profited from, given relatively low effort
It seems unlikely a lot of people didn't already know about this, it's hard to miss even if you only spend a few days with the EC2 API, and it's also quite surprising AWS have yet to correct the design. 90% chance it is mostly a UI problem -- there are no warning labels around snapshotting in the EC2 UI
> and it's also quite surprising AWS have yet to correct the design. 90% chance it is mostly a UI problem -- there are no warning labels around snapshotting in the EC2 UI
Snapshots are private by default, you have to actively make them public (impossible if encrypted) or share them (which also requires sharing the associated keys if encrypted.)
Now, AWS hasn't wrapped the extra layer of “by default, reject any setting or policy allowing public or cross-account access unless separate additional default switches have been toggled off” thing to EBS that they have to S3. But people still expose stuff via S3, so that's hardly a panacea. At some point, one has to conclude that customers are responsible, in many cases for giving too many(or just the wrong) people admin access to their accounts.
What I mean is https://i.ibb.co/P6N35qv/Screenshot-from-2019-08-10-10-27-25... does not make it clear whatsoever that 'public' really means public. Before we get into blaming the customer, there should be a bright red warning label in that dialog. Consider that English may not be the first language of many users reaching that screen
I think they have it for some stuff elsewhere, but it doesn't seem unreasonable to make public snapshots a per-account permission that defaults to disabled, and requires an interactive UI checkbox to enable. Out of the millions of AWS accounts, public snapshots are legitimately useful to maybe 1000 tops
All public EBS snapshots appear in the EC2 > Snapshots section in the AWS UI. Toggle the dropdown in the top right of the table to "Public" and you'll see them. Sort by size and you'll get some interesting looking ones at the top.
For EBS, step 1 is reading the docs, step 2 is cutpasting a documentation example.
For S3 I'm not sure how people are building their lists. AFAIK the API provides no enumeration. So this is possibly something coming from web crawl data (e.g. common crawl)
I just checked an EC2 console and I can see 19,356 snapshots created by other users.
I am so confused.
It would be trivial to make finding a snapshot require knowing a unique ID like an AMI.
And, why do I need to be able to search for 1000s of customers' public snapshots in the EC2 console? What conceivable purpose does that serve except being a giant opsec fail?
"a common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools" - Douglas Adams
But is the flawed system design here the automated system at AWS or the human-in-the-loop systems by which companies are providing admin access to IT resources, including AWS accounts?
Probably both! But I would argue that below a certain size provisioning things by hand probably makes sense. A UI that makes it too easy to make a private thing public is never ok.
Sure, but I don't think the current EC2 UI makes it too easy (and the S3 UI could only make it harder by not making public and cross account access possible at all.)
Once I mentioned on a mailing list Chrome’s reaction to mouse driver bug on my computer, it would buffer JavaScript events, and process it even for pages on a different domain.
Later I told the EFF I had a suspicion that iOS didn’t rate limit input events to the lock screen, independently a research found out about it a month later.
Even if there are zero days, I don’t think finding them is a particularly noteworthy or rewarding task.
I had a simple glance in the console and there are like 20,000 exposed ebs snapshots - available for anyone to copy and examine - I think that's only for a single region too - switch regions to see more.
Amazon should make an emergency decision to make all these private.
Sure it will break stuff but I'd be disappointed if Amazon left what is in effect a security hole open for the sake of backwards compatibility.
They should also give me a single click link when I sign in to show me all of my public ebs snapshots and throw it hard in my face when I sign in to the console so I simply cannot avoid seeing them all.
I have multiple AWS accounts and I just signed in to try to see if I have any public EBS snapshots and then I realised I would need to search every single region in every single account and then select every snapshot one by one to find out. That's a huge problem. I need a single click to show me every exposed snapshot across every region in my account.
UPDATE:
I can't say for sure if this is 100% right but I think if you sign in to your AWS account, then click on each of these links, you will find if you have public snapshots.
Maybe someone else could confirm if this is correct?
Never once have I ever found a use case for making public EBS snapshots.
Who on Earth is thinking that it is a good idea to take an EBS snapshot and make it public?
Note, several of those engagements did involve multiple accounts, and the need to share / copy AMIs and/or snapshots between accounts. But never making them public.