Hacker News new | past | comments | ask | show | jobs | submit login
Linus on kernel changes breaking user programs (from 2012/12/23) (lkml.org)
76 points by w1ntermute on Dec 27, 2012 | hide | past | favorite | 35 comments



Mauro's response shows incredible restraint. I commend him for that.

Linus could have easily written that email privately to Mauro and spared him the public shaming. He could have then written a stern yet civil public response. Would that be asking too much?

It doesn't matter who you are; basic human decency is not above you.


I've never worked at Microsoft, but I'm pretty sure Bill Gates and Steve Jobs sent messages like that to working groups, and even department lists. I have no idea whether it's right or wrong, but I don't think this kind of thing is atypical for big tech projects.

Linux just happens to be open source, so when it happens in Linux you hear about it.


All this argument seems to do is solidify the idea that engineers would be well served to work on their ability to interact with others. It doesn't make screaming and swearing at your peers okay because Bill Gates or Steve Jobs does it, as much as everyone would like to think so.


It's wrong. Common sense should tell you that it is wrong.


The kernel mailing list is by desision public, and Linus's email was part of an already public thread. We can dissagree with what he wrote, but I do not think he had a good option of writing it in private.


Two things: this should not have been the public chewing out that it was, Mauro obviously didn't do any of this on purpose and getting scolded like this privately would have been plenty, heck the guy probably berates himself a lot more effectively than this message could.

Second: the buck stops at the top.

Testing procedures should have caught the bug long before being seen in the wild so apparently linux still has some catching up to do there and that's a lesson that Linus should be learning here rather than to dump on the developer that caused the bug that wasn't caught in testing.

Process really does matter in cases like this. The problem is not that it happened, the question is how to make sure it never happens again. Maybe Linus was wearing his 'do I look like a people person to you?' t-shirt.


Second: the buck stops at the top.

Testing procedures should have caught the bug long before being seen in the wild so apparently linux still has some catching up to do there and that's a lesson that Linus should be learning here rather than to dump on the developer that caused the bug that wasn't caught in testing

The buck may stop at the top, but if Linus takes the public blame and picks it up, he then has to put his house in order, wouldn't you agree that's what we're seeing here?

Yes, testing procedures should've, would've, etc, but that's exactly why these kernel maintainers are there for. As far as Linux having some catching up to do, how often does this situation occur? I also believe the most valuable lesson to learn here is the one you can do nothing about: no matter how adamant you are about your dearest and most basic rules (we've heard Linus many times on code that break user-space), shit happens.

His frustration appears to stem from (rightful) presumptions that a kernel developer would not make such newbie mistakes.

I totally agree with you that this exchange should not have been so public, unfortunately kernel communication is by design. I would, however, also like to provide a different perspective. Linus is carrying the burden of being a bully, he does a dirty job of publicly berating people from whom he expects nothing but the best, everything about his handling of these situations shakes our moral fibre and makes him appear a dis-likable human being. But then, we're millions to enjoy the fruit of this dirty job everyday, blinding ourselves to what it takes to provide it. Such is often the burden of leaders.


> His frustration appears to stem from (rightful) presumptions that a kernel developer would not make such newbie mistakes.

Everybody makes mistakes including Linus.

His frustration appears to stem from the fact that the developer was blaming the user space program, rather than to immediately recognize the bug as a kernel problem.

But as a team leader you don't start yelling at people like that, this is unacceptable behaviour for someone of Linus' stature. It's immature and damaging to the community in the longer term even if in the shorter term it may have the desired effect.

Imagine you're working somewhere and your boss berates you in front of the whole world for some error.


I was conning a ship while the captain stood on the bridge-wing to watch our boarding party inspect a potentially hostile vessel. He was generally considered a really good, competent, and gentle captain. I was good at conning. I mean, throw a ring in the water, and I could maneuver the ship so the bos'un only had to stick his arm over the side and drop the grappling hook straight down through the ring to retrieve it. Every time. But this time, I had to keep the ship in a very specific position, to make sure the captain could see his men now on a potentially hostile vessel, we had intel that they were a mother ship for the cartel's cigarette boats. 450' ship vs a 150' ship, with 6 degrees of freedom each, rough seas, sundown, and I was off by 10 feet. A rather critical 10 feet: CO couldn't see inside their bridge. He blasted in front of my entire watch team. That was 14 years ago. I will never forget it.

I was in the OR with the coolest neurosurgeon I've ever seen. WashU grad (the med school's mean MCAT score is at 99th percentile for individuals). This guy could operate without disturbing the venous plexus around the spinal cord. I had a hard time focusing on the work because he was moving so little it didn't even disturb the smoke under the operating microscope. I could literally inspect individual specks of particulate smoke hanging in the air. He was moving microns at a time. He had asked for and rejected a specific pituitary in every surgery that week. This time, looking at a patient's dura mater, and trying to sneak around to the other side, seeing the wrong rongeur, his head didn't even move. But he grabbed the pituitary in both hands, bent the stainless steel, and flung it the floor so hard it bounced as high as the table and hit a wall. BA-WHONGGGG! The fact that it was this guy that did it is what everyone felt. I mean, it was palpable. That was 3 years ago. That scrub tech, I guarantee, will never forget that experience. I doubt I will.

I was taught years ago to carry two bags. Put the leadership tools you like in one bag. Put the ones you don't like in the other bag. Remember.

Sometimes the leader looks in the good bag, and comes up empty, or just decides that she's had enough.

Was it the right thing to do? It doesn't matter. Just decide which bag you're going to put it in.


Time critical engagements are different than software problems. You illustrate eloquently that there is a right time and place for everything. Linus' outburst was definitely not in the right place.

1) he's yelling at a volunteer

2) he's doing so in public when I'm sure he has the guys email address

3) (and this is the bit that bothers me) Linus seems to enjoy being abrasive at times

4) included invective that wasn't necessary at all.


People seem to be forgetting here that the problem wasn't that a mistake was made. We all make mistakes, and in fact the -rc series is part of the testing process; the mistake was caught before we cut the final release, so the development process worked.

The problem was rather the attempted justification of the mistake by claiming that perhaps it was the userspace program's fault. That was what set off Linus, and caused him to yell at Mauro. (It would be like someone trying to claim that perhaps the victim of sexual harassment was at fault. Sometimes, when someone blames the victim, the best response is a rather public chewing out of the person who tries to deflect blame from themselves and onto the victim.)


Your link-up with sexual harassment is in pretty bad taste, a piece of software isn't a victim.

And I can think of lots of different ways in which this could have been handled better.

Things like this are a learning opportunity, for everybody.

They're not a reason to let your self control go and start throwing abuse at people as though you're not an adult.


This is not a new message. Mauro knew this already. He just didn't internalize it. Sometimes, after saying the same thing quietly, and being ignored, you have to raise your voice in the hopes people will listen. Ultimately, of course, if people continue to refuse to listen, all you can do is fire them (which in the open source world, is to publically tell them not to bother to send you pull requests). Short of taking that drastic step, it's not like an open source maintainer can dock someone's pay.

In general, Linus only yells at people who should have known better. Mauro definitely fell into this camp.


He is right. If you are part of the linux kernel team you know what you got yourself into (Linus) and you are part of one of the highest profile opensource projects on this planet.

For are project like the kernel this is an appropriate response to eliminate errors. If they f..up on the development, alot of server will break all over the planet and many companies will get angry emails from people who have no clue why they cant upload pictures of their kitten.

The fact that you then try to make excuses for breaking user space, and blaming some external program that used to work, is just shameful. It's not how we work.

Most people would get some similiar response if you would do this on a clients project.


Have you seen Mauro's response? I think he took it like a champ. https://lkml.org/lkml/2012/12/23/87


Once you follow it all the way through, it seems to be a much more involved issue: https://lkml.org/lkml/2012/12/24/125


https://bugs.launchpad.net/ubuntu/+source/linux/+bug/607560 broken kernel could fuck up a lot lof users.I am suffering from this broken kernel, and I do appreciate what Linus is doing.



Using some logic here:

1 - It seems Mauro added a new error message called ENOENT; 2 - After this change, Mauro noticed an error at pulseaudio; 3 - Linus says that always the userspace is right and kernel space is wrong in those cases and simply discarded Mauro's argument;

But thinking about this I think Mauro is not entirely wrong. Mauro's question makes a lot of sense to me: Why pulseaudio failed after just a new error code was added to the kernel?

Can't it be that pulseaudio is doing some wrong treatment that is ignoring the possibility of a new (or non-existent) error code?


ENOENT isn't a new error code. It's the standard errno value that indicates an attempt to access a nonexistent file or directory on all POSIX platforms.

The problem is that it was being misappropriated in a nonsensical way, returned by an ioctl call. ioctl can only be called on already-open file descriptors, so it makes no sense for it to complain that the file doesn't exist. If the file pointed to by the file descriptor didn't exist, the file descriptor wouldn't have been able to be opened in the first place (errno would have been set to ENOENT by the call to open).


What if an ioctl is trying to indicate that a certain "entry" is not valid or available? Like when it is returning a cached entry or dealing with some container, where a notion of an entry does make sense. Insisting that ENOENT is reserved for path operations seems asinine.


What if an ioctl is trying to indicate that a certain "entry" is not valid or available?

I don't understand the question. It is not possible for ioctl to be called on a valid, open file descriptor that represents a file that is not valid or available. Even if the file is unlinked after being opened, the kernel will ensure that it remains available to any processes that have it open until they close the file descriptor.

Like when it is returning a cached entry or dealing with some container, where a notion of an entry does make sense.

I honestly don't know what you mean by this. ioctls, like nearly all Unix system calls, are file system operations, they necessarily and by definition refer to files.

Insisting that ENOENT is reserved for path operations seems asinine.

On the contrary, ENOENT has meant "No such file or directory" since the early days of Unix history. Arbitrarily deciding to redefine it now would be asinine. There are other errno values to represent other error conditions.


I am just saying that the abstractions that the drivers can implement, can be pretty diverse. A particular ioctl may very well be dealing with a container like abstraction (contained items may or may not be file paths), and some return value that indicates "NO SUCH ENTRY" (i.e. ENOENT) may not be so out of order.

These examples are contrived, but consider a driver returning a encryption key corresponding to a index, which is stored in some encryption hardware table (controlled by the driver). Or a driver could be maintaining a cache, which is indexed by a key. Or even routing entries. There are a million possibilities.

I have to admit that I have used ENOENT for cases where "NO SUCH ENTRY" made sense. But maybe the tradition is to exclusively use it for path based operations only. I better shut up before Linus gets on my case. Because it would not be good for the Linux world at all. I'd probably clock him really badly if he spoke to me like that.


As far as I know, the "ENT" in ENOENT refers specifically to a "dentry" (directory entry), i.e. the kernel data structure `struct dentry`, identified by a pathname in the virtual file system. It's not supposed to be a generic term for an entry in some arbitrary container of data.

I do see where you're coming from now, but I still think any usage other than the standard "no such file" meaning is errno abuse.


Seems reasonable enough. Can't help but wonder though, that a lot or the other errnos were probably also invented for very specific contexts, but find use in a wider sense because they capture a more generic error class.


Ah! So Linus argument makes sense indeed. No need to test if the file exists in a process that assumes it is already opened... Thanks for clearing that up.


My personal programming style when possible is to break - loudly and informatively - on any unexpected condition so that a human can fix it. A new error code meaning who knows what would qualify.

Now a widely distributed user library is probably not the right place for that style. But I cannot trivially disagree with a developer who took that route.


The original patch/commit that Linus calls out: http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6...


Whoa. That should have set off alarm bells when being reviewed. Changing errno return values just because? And a counter-change in another place to put the changed errno back to the original.

I'd accept that from an intern. Any grown-up that did it would get a talking to, though not in public.


Harsh, so god damn harsh!

Then again, I can't fathom how Linus must feel getting hundreds of pull requests that are plain wrong and/or stupid.


I've been defending Linus over this, but on second thoughts I think he crossed the line here:

> And you've shown yourself to not be competent in this issue, so I'll apply it directly and immediately myself.

I'm rather surprised at him; he's been pretty abrasive before, but seldom directly abusive like this.


Why self-censor 'f*cking' but not 'FUCK'? Consistency please.


This is awesome. I would entirely freaking love to be chewed out like that. I feel like this is what it feels like to be set back on the right track by folks like a young and tempestuous Steve Jobs, Bill Gates, etc. Sure he's cranky, but even through his crankiness you can tell he is fundamentally right.

DHH needs to swear more.


this is awesome, communication should be open (as in public) and candid. i'm not saying that i enjoy the approach, but there's much truth to linus' rant, that was a shit commit.


Torvalds swears in kernel chat, unrelated devs frown sternly, sun rises in the east, tides happen twice a day...




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: