Valid point, but I'm assuming that they never imagined 40-degree (outside) temperatures lasting multiple hours as a threat. Slough just got hit around 40.2 degrees, first time since records began.
It's also a bit of a stretch to call it "Google" when it's an Equinix datacenter. I know this is standard across all major cloud providers, and call me old-fashioned, but personally I'd prefer a bit more accuracy in naming of each region. Instead there is obfuscation. It's weirdly difficult to find a list of which datacenters correspond to which cloud region.
I hope there is a public postmortem that explains this. I have Cloud SQL instances with high availability enabled that have been down for over 5 hours with no workaround, when (according to docs) they should have failed over to the unaffected zones within minutes.
I think it's deceptive to say it's a single zone. We have VMs across three zones, but anything stateful on zone a is stranded, we can't get the volumes back. It's not just that the zone is down, it's that services are not easy to fail out of the impacted zone or move that's the problem.
Kind on implies regional isn't good enough to secure high availability. You need multi regional deployments. Good luck with that when the bill comes due.
I’ve just lost my evening due to the Oracle Cloud, not their fault to be fair, and availability was good, but our apps weren’t happy with the performance.
What's crazy to me is that a bunch of servers in AC cooled room consumes more power than entire villages in Europe suffering from this heat.
Imagine being on your phone tweeting, instagram stories, whatsapp messages in the blistering weather while the databases and servers receiving and delivering data sits comfortably in AC cooled buildings. Does anybody find this contrast just eerie?
Imagine if our workloads only did the minimum calculations necessary to achieve the associated user stories, instead of constantly rebuilding from massive dependency trees and all the other inefficiencies we put up with because it takes less dev effort that way and our performance budgets are being met either way.
Most places have emissions requirements for vehicles and so forth. Maybe some day we'll demand energy-efficient programming.
> constantly rebuilding from massive dependency trees and all the other inefficiencies we put up with because it takes less dev effort that way and our performance budgets are being met either way
I've been trying to minimise redundant calculations in our CI/CD workflows, but none of our tools seem to like that.
Docker rebuilds all downstream if one bit of a step changes, GitHub Actions doesn't let you cache on local runners and requires uploading everything to their size-limited cloud cache.
I'm looking for better ways but of course no service advertises their energy efficiency. That tells something about the industry's priorities.
I would take this further and find out what is the most energy intensive computing task across these DCs and attack those first. This is a huge unknown for me as I have no statistics on what activities result in the most energy usage in these DCs but if I have to guess, it is probably GPU/Database.
I would also tax personal data. Current regulatory framework strangely assumes that personal data has no value yet entire companies are built around monetizing and claiming it as theirs. Those companies are also the biggest consumers of these DCs.
Even by taxing personal data that is being used by companies, you could realize significant energy savings as they have to discern between what they need and what they dont. Asking for someones driver license to verify their account? Tax them yearly. Suddenly companies that dont really need it find themselves searching for more energy efficient solutions since their tax dollars depend on this activity.
Unfortunately, we may not see this happen for quite some time.
> I would take this further and find out what is the most energy intensive computing task across these DCs
Nearly all compute power in the world is used doing things that theoretically take no compute. The reason they use so much compute is it was easier to pay for that compute than pay some engineer to figure out how to use less.
Simple statistics can be deceiving. Just because there are no 40C+ temperature records at some location it doesn't mean that won't change next year. Southern Europe has been learning it the hard way for a while.
It's kinda bad, because cooling infrastructure when stretched can still cool some stuff, just less...
So they ought to be able to still keep some services up for some users.
They should have already practiced that as part of their emergency procedures, because they'd need to do the same is they had a severe limit on power use (eg. Due to a nationwide electricity or fuel shortage).
> So they ought to be able to still keep some services up for some users.
Wouldn't that depend on what exactly broke here? If the cooling of the power system failed, they might end up without power for example, impacting everything.
I suspect it wasn't that the cooling broke, but that the cooling couldn't keep up. Ie. The cooling could only remove XX kilowatts of heat, but that the equipment in a particular room was producing more than that.
I have to say, one thing about the U.S. is that there is AC everywhere - even up north. We have AC in elevators, on the busses - everywhere. Europe needs to start adjusting to the new normal of hot, hot summers.
Air conditioning consumes a ton of energy. About 10% of all electricity consumed worldwide. https://www.iea.org/reports/the-future-of-cooling It emits about 4% of greenhouse gases https://www.sciencedirect.com/science/article/abs/pii/S25424... But the volume itself is not meaningful - the gases used by AC units are extremely powerful - up to 750 times the greenhouse effect of CO2. And of course, the AC unit itself heats the city - which is why they get much hotter than the countryside during heat waves, further exacerbating the demand for AC and making the problem even worse. It's a vicious circle.
> if europe was hotter
How do you suggest achieving that? Other than global warming?
> the gases used by AC units are extremely powerful - up to 750 times the greenhouse effect of CO2.
They are sealed systems. They only emit gasses when breached. Or when not properly disposed of.
> And of course, the AC unit itself heats the city
A few watts - whatever power it requires to operate. Say it uses 1.5kw. That's the max it will heat up the city. It will of course move 4x this amount of heat(maybe more!), being a heat pump. But the 'new' heat is only what's imported by the power grid. The rest of the heat was already in the city.
It's also worth noting that it's often quite sunny when it's hot. If you power AC units with sunlight, you are actually not introducing any more heat to the system at all. That sunlight would already hit your city and be converted into heat, if you use some of it to move heat around, you are not increasing the overall heat level.
> They are sealed systems. They only emit gasses when breached. Or when not properly disposed of.
And mechanical devices having leaks is a folklore tale told to scare children?
> A few watts - whatever power it requires to operate. Say it uses 1.5kw. That's the max it will heat up the city.
Yeah. 1500W. Times the number of AC units in the city. That's what adds up to the temperature difference between the city and the countryside.
> If you power AC units with sunlight...
...with a solar panel above your own home, with no inefficiency in the electrical grid, the solar panel, or the AC unit itself. Please, this isn't a high school physics exam, you can't assume that there is no friction or that the AC unit is spherical. This is real life.
Take a look at heating if you want to see staggering energy consumption numbers. In a majority of places, people don't use heat pumps with a COP of say 4-5, but convert chemical energy with a cop of 0.5-0.9.
> It emits about 4% of greenhouse gases [...] But the volume itself is not meaningful - the gases used by AC units are extremely powerful - up to 750 times the greenhouse effect of CO2.
Do AC units emit these gases? Aren't you mixing up CO2 from electricity production and gases that stay internal to the unit?
They have two types of emissions: CO2 emissions from electricity production, and refrigerant gases that are supposed to stay inside the unit. But of course, units have leaks.
Good for you, but at least in the US, R600a (isobutane) is not used in AC units.
Besides, in the early XXth century, it was okay to vent CFC into the atmosphere. We didn't know that it destroyed the ozone layer. What makes you think that we won't discover something similar about isobutane in a few decades, besides wishful thinking?
It's a product of America's climate being more extreme on the highs and lows (a product of the jetstream). The one place that doesn't have extreme summers, the PNW, actually has very little AC too.
Aren't they adjusting customer thermostats, deferring plant maintenance, appealing people to conserve electricity and warning about blackouts in Texas precisely because AC is everywhere?
