Hacker News new | past | comments | ask | show | jobs | submit login
Meltdown and Spectre Linux Kernel Status (kroah.com)
217 points by dankohn1 on Jan 19, 2018 | hide | past | favorite | 101 comments



RHEL/CentOS state:

Red Hat built their own mitigation for Meltdown and Spectre and were the first distribution to have an updated kernel right when the embargo was lifted.

As far as I can tell, their meltdown mitigations are similar to what's in the upstream kernel now, and the Spectre mitigations relies on Intel's new MSRs (the microcode update that had since been retracted - new microcode_ctl packages remove it).

Interestingly, you can enable/disable the mitigations at runtime with Red Hat, which is useful for performance testing. I found no way to do this with the upstream kernel.

I'm not happy that Intel, Google, Red Hat and a few others have known about the bug for months and the upstream kernel devs are scrambling to build mitigations and still don't support the microcode mitigations. Google apparently patched their stuff in September, while everyone else is left out in the cold. This is a real mess. Surely there must be a better way than telling a few select companies?

The Red Hat guys in particular surely would've loved to collaborate with the upstream kernel. Must be frustrating not to be able to tell anyone.


> I'm not happy that Intel, Google, Red Hat and a few others have known about the bug for months and the upstream kernel devs are scrambling to build mitigations and still don't support the microcode mitigations. Google apparently patched their stuff in September, while everyone else is left out in the cold. This is a real mess. Surely there must be a better way than telling a few select companies?

I think Colin's take[0] was interesting and insightful there, especially in the light of him having handled a very similar issue a decade earlier:

> The way these issues were handled was a mess; frankly, I expected better of Google, I expected better of Intel, and I expected better of the Linux community. When I found that Hyper-Threading was easily exploitable, I spent five months notifying the security community and preparing everyone for my announcement of the vulnerability; but when the embargo ended at midnight UTC and FreeBSD published its advisory a few minutes later, the broader world was taken entirely by surprise.

> Contrast that with what happened this time around. Google discovered a problem and reported it to Intel, AMD, and ARM on June 1st. Did they then go around contacting all of the operating systems which would need to work on fixes for this? Not even close. FreeBSD was notified the week before Christmas, over six months after the vulnerabilities were discovered.

> […]

> To make things worse, the Linux community was notified and couldn't keep their mouths shut. Standard practice for multi-vendor advisories like this is that an embargo date is set, and nobody does anything publicly prior to that date. People don't publish advisories; they don't commit patches into their public source code repositories; and they definitely don't engage in arguments on public mailing lists about whether the patches are needed for different CPUs.

(emphasis his)

[0] http://www.daemonology.net/blog/2018-01-17-some-thoughts-on-... final section on vulnerability disclosures


I think lots of people (particularly security people) tend to fundamentally misunderstand how the Linux community works. It's not a single organization calling the shots where they can have a meeting behind closed doors about the vulnerability and internally incubate a patchset. That's simply not how Linux development works.

Linux is a loosely organized community of developers who collaborate in public. They don't have any ubiquitous communication channels established other than mailing lists. If you attempted to establish a private security mailing list, the list of people who get to go on it is (1) hard to define (2) very large and (3) frequently changing. The affected subsystems have lots of disparate maintainers and interested parties who need to get their eyes on any patches that go into them.

It might not be ideal from a security person's perspective but from everyone else's perspective this model has been the driving factor behind the most successful collaborative project of all time.


Linux has a security team/list that does handle disclosures and fixing of reported problems. It usually does this by just dragging in the responsible parties, but many times it just fixes the problems themselves and submits the patches to the proper part of the kernel.

For details on how this works, please see the kernel documentation itself: https://www.kernel.org/doc/html/latest/admin-guide/security-...


Thanks for clarifying, Greg.

>It usually does this by just dragging in the responsible parties, but many times it just fixes the problems themselves and submits the patches to the proper part of the kernel.

I think this is the key - once the patches are submitted they go through the normal workflow and aren't any different from a typical patch, right?


Yes, once the patches are generated, they get submitted just like any other kernel change, sent to the responsible subsystem and maintainer for inclusion like any other kernel change.


I think this is where the problems becomes apparent. The change then has to go through the subsystem's patch flow (in public) then into the merge window (in public) and sit in the release pipeline (in public) until the window closes and all of the rc's are through.

Personally I don't think this is a huge deal, but it's where the disconnect between the security person's ideal worldview and the reality of how Linux is built colllide.


Distros could speed this up. If the security team notifies RedHat, Ubuntu, etc and they can apply the patch for their kernels immediately. They probably need to backport it anyways, because no distro (well, maybe Arch?) just uses the latest Linux release.


FWIW, during the last year Fedora frequently had newer kernels than arch.


It seems to be a huge deal if it leads to the kernel folks both breaking the embargo and getting patches out later than they could have.


Only if you think either of those things are a huge deal either, which I don't.


Breaking embargoes generally leads to easy, packaged exploits appearing that many days earlier than they otherwise would have. Obviously patches being delayed gets you on the other end. How are either of those not a huge deal? Do you just think it's not important to have patched systems against published vulnerabilities?


> If you attempted to establish a private security mailing list, ...

Like the linux-distros [0] list? Or just the kernel developers?

(Note that both Linus and Greg are on this list.)

[0]: http://oss-security.openwall.org/wiki/mailing-lists/distros


Linux distros are different from Linux the kernel, and handle security differently. Greg has some good comments in this thread for how the kernel does it.


> It's not a single organization calling the shots where they can have a meeting behind closed doors about the vulnerability and internally incubate a patchset. That's simply not how Linux development works.

Bullshit. Ultimately, Linus decides what does or does not go in, assisted by the subsystem maintainers he picked. If they wanted to have closed doors meeting, they could, and would.

FreeBSD and the like have no more and no less "single organisation calling the shots", the only difference is that they're not a human clusterfuck.

> It might not be ideal from a security person's perspective but from everyone else's perspective this model has been the driving factor behind the most successful collaborative project of all time.

That just means Linux at large will ultimately get ignored for anything security-sensitive, and only specific distributions will be contacted.



Yeah so?

> The FreeBSD Project is run by around 500 committers, or developers who have commit access to the master source code repositories and can develop, debug or enhance any part of the system.

I doubt there are 500 people with push access to the main Linux repository. Again, the FreeBSD folks just have their shit together.


I was pointing out the existence of a governing body and a core team on FreeBSD, for which there is no analogue on Linux. There is no "the main Linux repository".


> there is no analogue on Linux. There is no "the main Linux repository".

Of course there is. Linus decides what goes into the mainline kernel, his repository is literally called "Linux kernel source tree"[0], with the stable "archive" being sensibly called "Linux kernel stable tree"[1], and Linux's analogue to the freebsd core team is de-facto Linus and his trusted maintainers, the people from whom he reviews and pulls commits.

[0] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

[1] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux...


You don't understand. This is a list of people who have ownership over some part of Linux:

https://github.com/torvalds/linux/blob/master/MAINTAINERS

Linus might have the "final say" but all of the work and knowledge is encapsulated in these people and they are generally given the autonomy to organize and maintain their subsystems as they see fit. As for Linus's tree, it's probably the most Linux Linux that's out there, but it's hardly the only one. Hardly anyone actually runs Linux from Linus's tree. Most distros maintain their own trees with their own patchsets.


> You don't understand

I perfectly understand that you're trying to excuse the utter clusterfuck that is the linux project's handling of security notifications, apparently on the grounds that they don't have their shit together and refuse to do so, ever.

> Most distros maintain their own trees with their own patchsets.

Yeah, and they can actually be trusted to not break embargoes.


Yes, at a macro level that is how it works, but practically due to having an enormous number of distributions there are many more forks. For example, the CentOS distribution maintains their kernel[0] separately from RHEL.

[0]: https://git.centos.org/summary/?r=rpms/kernel.git


That's not "the linux kernel" though, and distros are semi-separate actors which you can interact with independently from the baseline kernel — which happened here incidentally, Lima's original comment notes that Red Hat was contacted separately from the core kernel. Something which will likely be more common in the future if the mainline kernel team don't fix their handling of embargoes, they're incentivising security researcher towards having them discover vulnerabilities at embargo lifting along with the rest of the world.


If I'm understanding correctly, Colin is unhappy about two things:

(a) That Google didn't notify more operating systems about the vulnerability early.

(b) That one OS community who Google did notify early broke the embargo, leading to exploits circulating prior to the embargo date.

But these are two sides of the same coin. The more groups of people you tell about the vulnerability early, the greater the risk that someone is going to break the embargo.

For a bug like this, it seems like no matter how you go about disclosing things, somebody is going to criticize you for it. Either you didn't tell enough people and the ones you didn't tell are mad. Or you told too many people, and someone breaks the embargo and everyone is mad. Or in this case, both!


> But these are two sides of the same coin.

No, they are not.

> The more groups of people you tell about the vulnerability early, the greater the risk that someone is going to break the embargo.

Groups have different levels of trustability, disclosing to more trustable groups is less risky than disclosing to a single untrustable group.


Colin's disappointment with the Linux community implies he did trust them. On hindsight it's easy to tell "well, you should have disclosed it to all the people that wouldn't leak it, and only those".


I am hoping this is going to be one of the final nails into a coffin of "Google is so nice, google is awesome, google cares" crap that is still so popular. Google is a regular company. It behaves in its own and only own self interests. If it just so happen that for some reason its interests align with the interests of the others or something that we consider to be Good For Society[tm] only at that time Google will behave in the way.


Google has now found Cloudbleed, Heartbleed, Broadpwn, Spectre and Meltdown among others.

Off the top of my head can not think of any big vulnerabilities found in the last couple years not discovered by Google.

They pay engineers to find these and also pay bounties. This helps everyone including Google in the industry so would definitely still call them a nice guy. Maybe even a really nice guy?

It is very difficult with vulnerabilities what you share and when. Once Spectre and Meltdown was shared we have many PoC also out. So script kiddies all set up to exploit. So it is not simple but have zero idea how you could suggest some how Google is a bad guy here?

Also Google has also shared mitigations for Spectre that looks like a better approach to the broader community versus keeping as a competitive advantage.

They even shared with Amazon a company that has banned ALL companies on their marketplace from being allowed to sell some of Googles products. Personally I am not nearly that nice.


>Off the top of my head can not think of any big vulnerabilities found in the last couple years not discovered by Google.

Shellshock?


That's an interesting one.

Original discovery was by Florian Weimer. Many of the Shellshock family bugs were discovered[ß] (IIRC) by Michal Zalewski, after David Wheeler announced that bash was likely infested with a whole bunch of additional parser bugs.[0] MZ is Employed by Google.

Now, I think I recall MZ sharing at least some of the details of shellshock family bugs on his blog, so whether it goes under Google's umbrella or not might be subject to interpretation.

I believe it's safe to say that the original bug was not discovered by Google, but it's subsequent manifestations, after the first round of fixes, were.

0: https://en.wikipedia.org/wiki/Shellshock_(software_bug)

ß: can you say you "discovered" a bug if you are hunting on an already ripe ground with a fuzzer?


Why ß though.


Because I wanted to avoid a naked asterisk. Index for references, something else for lesser footnotes. And no possibility to collide with emphasis parsing.


Not as a defense of Google, but what's so wrong about Google not contacting FreeBSD? It's obvious that not all operating system vendors could be contacted (should they be contacting small hobbyist operating system projects? What about small niche commercial ones?), so choices of who to contact and when had to be made.

If FreeBSD was contacted early on, would we be hearing these gripes from some NetBSD or Dragonfly BSD developer? Maybe that's actually a reason BSD was contracted late, maybe they expected interested scrutiny from the other BSD projects from any kernel patches. Linux certainly ended up exhibiting that behavior, maybe they were worried about it there too but Linux's size meant a different call was made?

I understand why BSD devs are upset, but the complaining comes across as exactly what it is, whining they didn't get treated like the big boys. It sucks. Linux used to be right there with you, and neither got any special treatment, but times have changed.


> Mayvethat's actually a reason BSD was contracted late, maybe they expected interested scrutiny from the other BSD projects from any kernel patches. Linux certainly ended up exhibiting that behavior, maybe they were worried about it there too but Linux's size meant a different call was made?

FreeBSD is a serious project which takes security embargoes seriously, they don't break embargoes: http://www.daemonology.net/blog/2018-01-17-some-thoughts-on-...


> Not as a defense of Google, but what's so wrong about Google not contacting FreeBSD? It's obvious that not all operating system vendors could be contacted (should they be contacting small hobbyist operating system projects? What about small niche commercial ones?), so choices of who to contact and when had to be made.

> I understand why BSD devs are upset, but the complaining comes across as exactly what it is, whining they didn't get treated like the big boys. It sucks. Linux used to be right there with you, and neither got any special treatment, but times have changed.

Not notifying the FreeBSD devs is bad for the entire internet. The Netflix open connect CDN is FreeBSD and pushes 37% of North American downstream traffic[1]. Limelight one of the big CDN providers use FreeBSD for it's edge network[2]. Some of the DNS servers for .com and .net are FreeBSD[3] Juniper routers run FreeBSD[4] PS4s run a modified FreeBSD[5], the Apache foundation runs a lot of there infrastructure on FreeBSD[6]. Whatsapp with it's billion plus users run on FreeBSD servers[7].

None of those things should be vulnerable to known bugs if we want a safer internet. So notifying the FreeBSD devs 6 months late did the entire internet a disservice regardless if you use FreeBSD directly or not.

[1] http://appleinsider.com/articles/16/01/20/netflix-boasts-37-...

[2] https://www.freebsdfoundation.org/testimonial/limelight-netw...

[3] https://www.freebsdfoundation.org/testimonial/verisign/

[4] https://www.juniper.net/documentation/en_US/junos/topics/con...

[5] https://www.theregister.co.uk/2013/11/16/sony_playstation_4_...

[6] https://www.freebsdfoundation.org/testimonial/the-apache-sof...

[7] https://www.freebsdfoundation.org/testimonial/whatsapp/


You know what almost all those examples have in common? They don't generally run user code, so are very unlikely to be running non-vetted code. I wouldn't be surprised if the majority of them don't bother with the fix because of the low chance of being affected and potential performance cost.

When it comes to operating systems that have a large base of users and may run unvetted code, the big candidates for that are operating systems that have home users downloading and running software or in the browser, and shared environments such as shell servers, shared hosting providers and virtual services providers. FreeBSD's common industry use cases and low end-user use mean it's much less exposed to this particular problem.

Edit: grammar, typoo and clarity fixes.


In this particular case, I agree, but it does suggest there should be a workflow for notifying the FreeBSD team of bugs, and I don't see what harm using it here would have caused.


I think the harm is in increasing the parties that have to know about and conform to the embargo.

If FreeBSD should be notified in this case, even though they aren't really in most of the high impact areas, should OpenBSD be notified? What about QNX? What about OpenSolaris? Apparently Joyent got late notification as well[1], and since they actually are a cloud provider, I think they have a much better case for being notified before FreeBSD.

In the end, if there's an information embargo there needs to be a decision about who's in and who's out, and most times there will be some excluded party that really wishes they were included. You can't make everyone happy, but I don't think FreeBSD really needed a huge advance warning, given what it looks like Google's criteria was for notification. The fact the hyperthreading bug that keeps being referenced was handled differently seems to me to be mostly an artifact of a BSD developer finding the bug.

1: https://help.joyent.com/hc/en-us/articles/115015938847-Secur...


That's correct -- we at Joyent had no advanced disclosure. That we have our own operating system (SmartOS) that also serves as the hypervisor in our own public cloud (and that Intel knew and knows all of this) really did/does leave us in a uniquely terrible position. While we have made excellent progress on KIPT for SmartOS/illumos, it is also true that this is very delicate work in the core of the system; that we are forced to do this while vulnerable belies any notion that this was responsible disclosure.


Ah, that's unfortunate. From a little research, the wording from Linode's announcement makes it sounds like they didn't get advance warning, but the wording from Digital Ocean's announcement sounds like they had a few days. I'm not sure how it all worked out, but I agree you guys are a specific case where at least little advance warning would have helped quite a bit, since you can't piggyback on the community nearly as much. Linux providers can at worst just wait for kernel patches and then roll them into production.


> I think the harm is in increasing the parties that have to know about and conform to the embargo.

FreeBSD has conformed to embargoes in the past, telling them would not harm the embargo.

> but I don't think FreeBSD really needed a huge advance warning, given what it looks like Google's criteria was for notification.

You have source for their criteria?


> FreeBSD has conformed to embargoes in the past, telling them would not harm the embargo.

So? Should they be notified of Microsoft windows vulnerabilities just because they've shown they can handle themselves? There has to be a cutoff somewhere. I don't think FreeBSD was in the highly affected subset of use cases. I can imagine others coming to the same conclusion.

> You have source for their criteria?

No, that's what "looks like" in my comment is there for, to imply it's an assumption on my part. They notified chip makers, the large end user OS makers, and many/most the major cloud providers (this probably depends on what you consider major, I don't know if Linode or Digital Ocean were notified, but AWS was, and Joyent wasn't..), which are what I would think the highest impact areas are, as those use cases either require running third party code or often due so (end users and downloaded software or browsers with JS).

To me, that appears like they tried to get the most major players most affected some info.


> So? Should they be notified of Microsoft windows vulnerabilities just because they've shown they can handle themselves? There has to be a cutoff somewhere. I don't think FreeBSD was in the highly affected subset of use cases. I can imagine others coming to the same conclusion.

Seriously what the hell are you talking about? I was saying FreeBSD should be notified of security bugs that effect FreeBSD and not 6 months after other Operating systems are told. I said nothing of Microsoft.


Look at it this way, what's the common case that a FreeBSD install could fall victim to this? Are people commonly running untrusted binaries on those desktops or servers? Is FreeBSD used for a large amount of shared hosting platforms (larger than Linode/Digital Ocean)?

Of those cases where it is more susceptible, what percentage of the overall likely susceptible hosts does that account for? Do you think it's even 0.1%? Even 0.01%?

If that's the case, and I was Google, I'd be just as likely to notify FreeBSD about this when attempting an embargo as some random MS vulnerability.


Your seriously overthinking this. It's as simple as FreeBSD is used extensively across the world, not as much as Linux or Windows, but still millions of servers and appliances run FreeBSD. The next detail relevant is whether FreeBSD devs honor embargoes, which they do. Everything else you have written is just trying to rationalize a bad decision by Google and company.


You act as though they don't run code exposed to the internet that could be used to leverage this privilege escalation. Talk about short sighted...


It would have been all well and good if at the same time announcement was released a new linux mainline build with the mitigations would have been available. It has not been. Only Google's systems got the patches all the way back to September.

I'm not at all anti Google saying "We are Google. If you pay us to run your systems you would be protected. If you do not, then eff you, we don't care". I'm anti-fluffy "Google is awesome and does things for the community" garbage popular in tech circles, HN included.


Google found these vulnerabilities in the first place, and many serious ones before Spectre/Meltdown too. We don't know if state or other actors found them first, but we know that if Google was not in fact "doing things for the community" here by finding and disclosing them, they'd still be unknown to us, and there would be no mitigations at all. To rubbish all of that defacto contribution, because they didn't disclose it the way you might prefer, is asinine - talk about looking a gift horse in the mouth!

All you're really doing here is confirming that you are typical of the HN userbase these days (far past its prime and filled with reactionary tinfoil hat types with a raging hate-on against one company or another) - that Google is on your personal shit list, and that you'll characterize just about anything positive about them as "fluffy garbage". And that's just not very interesting - it's tiresome, practically-writes-itself noise to everyone other than the choir you are preaching to.


> we know that if Google was not in fact "doing things for the community" here by finding and disclosing them, they'd still be unknown to us, and there would be no mitigations at all.

Not only is that a hypothetical what-if scenario that you've invented whole-cloth to support your argument, it's provably false since there was parallel discovery from three other teams at approximately the same time.

https://www.wired.com/story/meltdown-spectre-bug-collision-i...


Not at all. I'm using google products and I'm paying for Google products with real $$$, not fluffy feelings. I also use products of Microsoft, and I pay for products of Microsoft with real $$$, not fluffy feelings.

I am prefectly OK with Google and Microsoft's position being "Eff you, we only support our customers and those who are not our customers can become our customers so we can support them".

The general view on HN for some reason is that Google this benevolent do gooder. That's the view that needs to be dispatched. Google is a standard company. It happened to be quite big and have quite a big market. To defend its market it needs to be involved in security research. For lots of reasons, of which I would say PR is one of the most important, it also cannot afford to sit on the vulnerabilities that it discovers forever as someone else may discover them independently ( think Chinese competitors of Google ) and may use them against Google's interests in a way it cannot currently identify.

HN user base gets lots of Googler who are pretty fanatical about the company -- which I guess is understandable as without Google they may not be able to support their lifestyle.


You realize there are several fortune 500 companies whose products are based on FreeBSD, right?

The same cannot be said for netbsd or dragonfly. To call FreeBSD a "hobbyist OS" shows a distinct ignorance of the market.


Are you contending that those products often run unvetted code? There's a world of difference between the likelihood of these problems being exploited on a servers that aren't shared, routers, small internet devices, and IoT devices compared to shared hosting environments, end user systems (whether they be mobile or desktop) and cloud/virtual server services. Of the latter, I'm o ly aware of FreeBSD being used in any frequency for shared hosting, and even that is relatively rare. FreeBSD just doesn't have the same exploit exposure as the others. But Google compute cloud, AWS, windows, MacOS, and Linux (including Android) definitely are ripe for exploitation.


What do you mean "run unvetted code" - when you've got network facing service, there WILL be remote exploits available. This bug can then be used for privilege escalation. Your understanding of the FreeBSD user base and it being "relatively rare" is so far off base I'm not sure what to tell you. For starters I'd be willing to bet the number of Juniper switches and routers in the wild number in the tens of thousands and they all run FreeBSD. EMC has several storage products. There are multiple firewall vendors. NetApp has several storage products. Etc. etc. etc. Almost all of which are exposed to multiple users, and while some may be behind a firewall, that's little comfort for targeted attacks...


This isn't about FreeBSD not getting a fix, it's about not getting advance notice. If to be exploited you require a remote unprivileged exploit first, I think you are at least an order of magnitude, if not multiple, less likely to be exploited than a cloud vps provider or desktop user that is downloading and running programs from non upstream sources.

I mean, should OpenBSD have received advance notice? What about QNX? What about all those embedded POS system providers? FreeBSD accounts for a very small percentage of high risk systems, and I think that explains this.


As soon as changes hit the mainline kernel that started the clock. The blogpost that made this all public was just someone noticing the kernel changes and speculating about how they must be in response to an embargoed security bug.

So no, there's no better way. You tell every organization that's capable of patching their kernels privately and keeping the vulnerability under wraps - something that mainline Linux can't or won't do, for better or worse. This is like complaining that you found something in the last place you looked - the linux mainline were the last people who heard about it before it went public because as soon as the linux mainline heard about it, it became public.


I'm not going to beat around the bush here, there was speculation about what might be behind this sudden push for isolating kernel page-tables, but what really blew the lid was AMD outlining the specific details of the vulnerability in a pull request.

Everyone here acted responsibly except AMD, who broke embargo so they could force a response from Intel at CES and sell more processors.


So… the better way is to just not contact the Linux mainline folks, as contacting them is the same as lifting/breaking the embargo.


They really could have done a tiered release. Keep the knowledge of the vulnerability as close as possible for the ~6 months they had it, then release it to a greater (but still restricted) set of people for another ~6 months.

the worst case would have been someone in the latter set leaking the info, which is just what we have now. So it could have been potentially better. There was really no reason to drop a bomb like this.


It didn't survive even the first 6 months and was published shortly before the embargo was over.


Nitpick: it actually survived 7 months. From June 1st to January ~1st


I'm afraid this is the kind of problem where there is no easy solution. Is there any specific improvement you would like to propose?


> I'm afraid this is the kind of problem where there is no easy solution.

Issues on a similar scale have been properly handled in the past: http://www.daemonology.net/blog/2018-01-17-some-thoughts-on-...

> The way these issues were handled was a mess; frankly, I expected better of Google, I expected better of Intel, and I expected better of the Linux community. When I found that Hyper-Threading was easily exploitable, I spent five months notifying the security community and preparing everyone for my announcement of the vulnerability; but when the embargo ended at midnight UTC and FreeBSD published its advisory a few minutes later, the broader world was taken entirely by surprise. Nobody knew what was coming aside from the people who needed to know; and the people who needed to know had months of warning.

> Contrast that with what happened this time around. Google discovered a problem and reported it to Intel, AMD, and ARM on June 1st. Did they then go around contacting all of the operating systems which would need to work on fixes for this? Not even close. FreeBSD was notified the week before Christmas, over six months after the vulnerabilities were discovered.


> As far as I can tell, their meltdown mitigations are similar to what's in the upstream kernel now,

I haven't checked, but I'd be surprised. Basically all the backports except 4.14's are quite different from upstream's.


If you also want to know if intel has released a microcode update for your CPU yet (regardless of whether or not it's available in your distro yet):

Download the latest archive from Intel at https://downloadcenter.intel.com/download/27431/Linux-Proces... (currently microcode-20180108.tgz) then:

    tar -xzf microcode-20180108.tgz
    sudo apt install iucode-tool # or whatever your package manager is
    iucode-tool -tb -lS ./intel-ucode/*
This will crosscheck the microcode binaries with the signature of your CPU and output something like:

    001: sig 0x00010676, pf mask 0x01, 2010-09-29, rev 0x060f, size 4096
    002: sig 0x00010676, pf mask 0x04, 2010-09-29, rev 0x060f, size 4096
    003: sig 0x00010676, pf mask 0x10, 2010-09-29, rev 0x060f, size 4096
    004: sig 0x00010676, pf mask 0x40, 2010-09-29, rev 0x060f, size 4096
    005: sig 0x00010676, pf mask 0x80, 2010-09-29, rev 0x060f, size 4096
As you can see Intel hasn't released any updates for my old CPU (latest is from 2010)


A word of caution; some of the microcode updates have apparently been causing reboots [1].

[1] - http://www.zdnet.com/article/meltdown-spectre-intel-says-new...


Yup

Although you can also install microcode with the iucode-tool, the example above only checks the downloaded archive.


So this microcode goes in /lib/firmware/intel-ucode (in ubuntu) will this then firmware update the CPU with the latest microcode, or only use this like a driver leaving the CPU alone?


Microcode is loaded into some on-die volatile RAM every boot by the CPU... there is a CPU instruction to do it. AFAIK:

First the CPU loads an on die ROM of the microcode it was released with on it's own. Second the BIOS/UEFI may load an update into the CPU before boot if it has a newer one. Third linux and some other OS will load any newer microcode into the CPU at the bootloader stage before the kernel loads.

This is also commonly the case for other modern hardware, not just CPUs... although I expect it doesn't need to happen before the kernel loads and is probably bundled into their respective drivers.


Am I correct in stating that the microcode update recently released by Intel was buggy and therefore pulled back by Linux distros and Intel themselves, and also a fixed version of the microcode update has not yet been released by Intel for Linux as well as other OS like Windows?


I am curious to know more details on the status of the microcode update specifically when it comes to Windows Update.


I do not believe the Windows patches include microcode at this time.


What a mess this has been so far... I appreciate all the effort kernel folks had put in it until now but the whole tier 1 companies cabal thing is just disgusting and it made the problem worse with the early disclosure (not to say "leak" if you count LKML diffs flying around). If at least all the cabal parties were equally covered and protected by now, but nope nope nope. If I were a kernel developer not working for any cabal party I would be very pissed off.


I recall Herb Sutter saying that companies collaborating on the C++ standards had to do it via an organization like the ISO to avoid the risk of antitrust(?) charges.

I wonder if the "cabal" you mentioned actually broke those laws?


I know it sucks, but I think the intentions were good here. By managing the disclosure in this way, the majority of users _should_ have been protected before public disclosure. Of course that didn't happen, but that was the intent IMO.


How would you do it then? Publish as soon as it's discovered, and let everyone scramble for fixes as its being exploited?


Notify groups who have to develop fixes (in this case: OS, compiler developers). Especially when they have a record of not violating embargoes and good faith in fixing issues.

When the longer time for fixes is done, post an announcement in private pre-disclosure lists.

For a very complicated change, give a few days of private testing and provide patches and details to groups that must apply the patches (oss-sec distro list, etc.)

For a simple one with existing backports, notify that <DATE> is publication date for a type of vulnerability to <PROGRAM>.

Wait 2-4 weeks to publish a working exploit.

Don't notify bodies who don't have their own custom code that must be fixed, but happen to pay you enough money.

The latter is actually what happened in this case and it's immensely frustrating.


0. fucking put the pressure on Intel and other chip makers to have microcode updates ready by NYE, like, you know, 6 months after the first report

1. make sure all tier 1 companies involved are equally patched and on the same page as far as mitigations go before EOY, then involve tier 2 companies (see news on "the impromptu war room") a full month before the embargo deadline

2. not commit or patch anything or fly commented diffs around in the freaking public LKML; git branches to be tested and secondary mailing lists can be private if only for a brief period of time

That's definitely not what happened, and probably the dirty details have not surfaced yet. I can't wait for this to be reasonably over so we can start to read post mortem on the whole process.


You can't really force a different company to have patches and updates by a certain deadline, can you. The only hard leverage is the officially planned disclosure date.


My job is just to backport the fixes into an olde kernel version, but I'm not pissed off :p I don't believe in secrets not leaking. The more parties you tell a secret, the sooner it will leak. And if it had gotten leaked, I'd be in a worse situation than right now.


I’m confused… The article led me to recheck whether I’m loading the newest Intel ucode update during early boot, and it turns out I am, but the latest ucode revision for my Ivybridge machine is from 2015. Didn’t Intel push microcode updates for all affected systems yet, in preparation for more spectre-related mitigations?


Lenovo mentions [1] a "target availability" of a bios update for my x230 with ivybridge for 2nd of February. Intel didn't update the microcode for those chips yet.

[1] https://datacentersupport.lenovo.com/de/de/products/storage/...


Intel is behind on the patches... 6 months is not enough time apparently.


So, apparently my system is unpatched. What is the proper thing to do now?

Should I immediately update my BIOS (would this also patch my CPU's firmware?) or just wait for mitigations arriving with OS updates?


> Should I immediately update my BIOS (would this also patch my CPU's firmware?)

The current recommendation is "no", since some of the microcode updates are known to cause stability problems. It's better to wait for Intel to fix that first, then make sure your BIOS update has a microcode version without the stability problems, then update. Yes, it's a mess.

> or just wait for mitigations arriving with OS updates?

Even after you update the microcode, you still need a OS update to enable the microcode mitigations, so you have to wait for a OS update anyway.


The current recommendation is "no", since some of the microcode updates are known to cause stability problems. It's better to wait for Intel to fix that first, then make sure your BIOS update has a microcode version without the stability problems, then update. Yes, it's a mess.

Note that the microcode update does not have to be delivered through a BIOS update. Linux can also update the microcode during early boot:

https://wiki.archlinux.org/index.php/microcode


This command does not show anything on my ubuntu 16.04 lts machine.

grep . /sys/devices/system/cpu/vulnerabilities/*

is there an alternate command for ubuntu?


for ubuntu command check here: https://askubuntu.com/questions/992137/how-to-check-that-kpt...

Here is one of the commands recommended in that thread:

grep -q "cpu_insecure\|cpu_meltdown\|kaiser" /proc/cpuinfo && echo "patched :)" \ || echo "unpatched :("


That line worked for me, but do note the first command listed in the answer you linked to is NOT reliable and is not always consistent with the next two commands.


This command appears to work in Fedora too...


> Some “enterprise” distributions did not backport the changes for this reporting, so if you are running one of those types of kernels, go bug the vendor to fix that, you really want a unified way of knowing the state of your system.


Having actually read the article, this would suggest that your kernel is too old to have the fixes.


Not true. Ubuntu packported the kernel patches. 16.04 should be updated automatically as long as you've restarted your server


To be clear, Ubuntu's Spectre fixes are only in -proposed right now. They are scheduled to be moved to the regular repos on the 22th.


Just to follow up, looks like Spectre fixes are now available but only for Variant 1. The following is on a 16.04 VM running HWE 4.13.0-31. Additionally, reptoline patches are not available.

  CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
  * Checking count of LFENCE opcodes in kernel:  YES 
  > STATUS:  NOT VULNERABLE  (114 opcodes found, which is >= 70, heuristic to be improved when official patches become available)

  CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
  * Mitigation 1
  *   Hardware (CPU microcode) support for mitigation
  *     The SPEC_CTRL MSR is available:  YES 
  *     The SPEC_CTRL CPUID feature bit is set:  NO 
  *   Kernel support for IBRS:  YES 
  *   IBRS enabled for Kernel space:  NO 
  *   IBRS enabled for User space:  NO 
  * Mitigation 2
  *   Kernel compiled with retpoline option:  NO 
  *   Kernel compiled with a retpoline-aware compiler:  NO 
  > STATUS:  VULNERABLE  (IBRS hardware + kernel support OR kernel with retpoline are needed to mitigate the vulnerability)

  CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
  * Kernel supports Page Table Isolation (PTI):  YES 
  * PTI enabled and active:  YES 
  * Checking if we're running under Xen PV (64 bits):  NO 
  > STATUS:  NOT VULNERABLE  (PTI mitigates the vulnerability)


You can find a "shared by a subscriber" non-paywall link of the lwn.net article mentioned in the article here:

https://news.ycombinator.com/item?id=16177525


LWN is worth supporting.


Sure it is. They allow the link to be shared nevertheless.

https://lwn.net/op/FAQ.lwn#slinks

Where is it appropriate to post a subscriber link?

Almost anywhere. Private mail, messages to project mailing lists, and blog entries are all appropriate. As long as people do not use subscriber links as a way to defeat our attempts to gain subscribers, we are happy to see them shared.


And as this is a blog entry, I think it's a place where the author should have used the subscriber link.

> As always, [lwn.net covers the technical details](https://lwn.net/Articles/744287/) about the latest state of the kernel patches to resolve the Spectre issues, so please go read that to find out that type of information.

The non-subscriber link in the blog entry results in a paywall. I think LWN would prefer that Greg used a subscriber link in this case. You may be being downvoted because people think you're trying to post a paywall workaround, but I agree that this would have been an appropriate place to use a subscriber link.


You can use a subscriber link, but Greg chose not to. Jon Corbet is a mutual friend, and LWN is essentially a two person shop (Jon and Jake do all of the work, and they are not getting rich off of what is essentially a labor of love).

So if someone decides they want to use a somewhat more proactive way of encouraging people to subscribe to LWN, personally I think it's appropriate and its his choice. Just as someone who wants to post a subscriber link to Hacker News is also appropriate. (But you should seriously think about subscribing, even at the Starving Hacker level, if at all possible. I subscribe at the highest tier even though my company has a corporate subscription, Just Because.)


I've spoken to Jon Corbet, publisher of LWN, about sharing subscriber links on HN. He's fine with it. Here's the link Greg KH references:

https://lwn.net/SubscriberLink/744287/ec63c9aa1fe236c7/

Please get your company to subscribe to LWN. It's an amazing resource and needs your support.


> And yes, it is behind a paywall for a few more weeks, you should be buying a subscription to get this type of thing, go do that!

LWN has tons of great content and deserves any support they can get.

But I strongly suggest that in this case, there shouldn't be a paywall. The current state of Meltdown/Spectre fixes is of high public interest. And people who are normally not into kernel development should be able to read this information without having to pay (or find workarounds).




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: