Seth Roberts has made his entire reputation on self-experimentation. (Partly on the strange things he claims to have learned by doing it, and also partly on his defense of it as a methodology.)

He's written some papers that talk about the methodological concerns. Here are two that Google churned up:

http://sethroberts.net/articles/2010%20The%20unreasonable%20...

http://escholarship.org/uc/item/2xc2h866

I can't honestly claim that he has successfully made n=1 experiments respectable, but I think it's safe to say he's very well aware of the criticism on these grounds and is not ignoring it.

This is interesting and I'll have a look at them, though in this case my issue isn't with n=1, it's with the wider design of the experiment and the failure to be clear about it's limitations.

The guy is a prof. of psychology at berkeley, and knows his statistics much better than most researchers in medicine or nutrition.

I don't respect your skepticism, unless you are just as skeptical about advice you get from your Doctor or pharmacist.

Roberts looks for effective, simple treatments that you can try yourself without requiring a large budget or large lab, publishes everything without trying to make money of it -- including a diet-free diet that works remarkably well, that flies in the face of all "nutrition science" common sense. He is not the crackpot you think he is.

I'm sceptical about things in so far as you can be and actually achieve stuff - obviously certain things have to be taken at face value because the effort in not doing so massively outweighs the benefits you might gain from a more detailed assessment.

But if you read an article (as in this case) which has obvious flaws, not to point them out just because you can't research every drug or treatment a doctor gives me would clearly be absurd.

And I'm not saying he's a crackpot, I'm saying that in this instance he's carried out an experiment with a flawed methodology and drawn overly strong conclusions from it. The can be a (former) professor at Berkeley or a high school drop out, it doesn't change the weaknesses in what he's written in that article.

All due respect, I don't think you're getting what Robin Hanson is about. (Which is understandable if this is the only post of his you've ever read.)

The point of this post isn't to criticize Alex Grass's choices, it's to point out that our social system has a rather bizarre bias. We applaud Grass's spending money on these charities -- presumably because it's an ostensibly selfless act -- despite the fact that he actually did more social good by making the money than he did by giving it away. (Not saying I agree, just trying to clarify.)

A running theme, maybe the primary theme, in this blog is that people make deeply suboptimal choices (whether measured by social interest or self interest) because they're really more interested in sending signals than they are in achieving optimal outcomes.

The blog is worth a longer look; for me, I don't know how much I agree with Hanson's signaling-centric view of the world, but I'm impressed with how deeply he thinks about these things, and how he manages to talk about these topics without devolving into misanthropically simplified "people are dumb" conclusions.

You are correct in assuming that I've read very little Robin Hanson. I'll definitely give it a further look.

I've just become tired recently of every large (and usually socially beneficial) charitable donation being followed by a wake of "how could he/she have used his/her oversized wealth for that cause when there are so many more worthy causes that I care about?"

The point is not that the money went to a valid cause that the author doesn't care about; the point is that the money went to an invalid cause. Which is fine and legal and all, but it's not really charity.

Actually, upon further reading, I think ck113 has got it dead on. Its not about the money, the individual or even the charity where the money actually ended up. Its about the bizarre social pressures that drive the decisions.

Wish the blog post had spent a few extra sentences setting up some context for those of us who are less "Hanson literate".

My question is whether the signalling is actually approaches a deeper optimal.

For example, I seem to be more deeply motivated by things like relationships with others and ideals, than naked self interest or societal interest. These suboptimal choices Hanson points out signal to others the value of the less physically based motivations for themselves. I start to lose hope if I think I'm the only one who has such ideals and somewhat question my own sanity. But, when others signal that these ideals are actually crucial to their own decision making, I feel I'm not the only one and am more motivated to follow them.

But, how do such ideals related to naked self and societal interest? Well, the ideals tend to motivate the deeper insights we generate, and motivate people to work together voluntarily without the overhead of contracts and reward/punishment systems. I would argue that while perhaps inefficient in the short term, these "biases" actually turn out to be much, much more efficient in the long term. I.e. just look at what happens when people do lose hope. They don't suddenly become productivity monsters, instead they tend to turn both self and other destructive. Definitely not creating value in terms of self and societal interest here...

I think it's interesting. Not so much the ultimate question of whether they can find him, but rather the things we learn along the way about exactly how much a reader base with no special (e.g., law enforcement) privileges can dig up about someone they've never met. Not just general character facts about him (he's left handed), but also time- and date-stamped activities (he took a picture of his cat at this time and this place).

It would be a lot more interesting if each bullet point of information came with a link to a 'trail' describing how that factoid was uncovered. (For example, I'd be particularly interested in learning how we know about his ATM activity. Assuming no one got a court order, how we anyone get access to that info?)

he gave his editor access to his bank accounts before he left, and the editor has been posting data dumps regularly to the wired site.

Ok, I know it's violently tangential and pretty silly, but I noticed this in the article too. I thought it was interesting because (I'm guessing) it's not a case of the author not knowing the grammatical rules you've just described, but of his not knowing the saying he's quoting.

The phrase is "no wine before its time." Not as in "before it is time", but as in "before that wine's appointed time." It was the slogan of the Paul Masson winery, though I can't find a great source for that. (Maybe this one: http://wineeconomist.com/2009/02/10/no-wine-before-its-time/)

So we do want the possessive here, not the contraction.

That makes the phrase make a lot more sense.

From the article: "Another interesting feature of NP problems is that a solution can be checked in polynomial time by a deterministic Turing machine. So, checking a solution of a NP problem is a P problem."

That's not just an interesting feature of NP problems, it's an alternate definition. You can define NP as the class of problems that have polynomial-time "verifiers". That is, the class of problems for which, given a candidate solution, you can determine in polynomial time whether that solution is correct.

Examples: given a proposed assignment to the variables of a boolean formula, you can check in P-time whether that assignment satisfies the formula. Given a proposed route through a set of cities, you can check in polynomial time whether that route has length less than K. Given a set S of integers, a subset of S, and a target T, you can check in P-time whether that subset sums up to T. Etc.

I find the "polynomial time verifier" definition of NP to be much clearer than the "solvable by a polynomial-time nondeterministic Turing machine" definition. They're exactly equivalent -- I don't know why the latter is so much more common.

Ooh! Ooh! I just remembered another cool thing about the "polynomial-time verifier" formulation of NP. It makes it possible to phrase the P vs. NP question much more starkly.

The standard formulation of P vs. NP is something like "do deterministic polynomial-time Turning machines have the same power as nondeterministic polynomial-time Turing machines?" A deep question, but not a very snappy sentence.

But the alternate formulation is something like: "is it always as easy to find a correct answer as it is to check if a given answer is correct?" Really brings home why it seems so painfully obvious that P is not equal to NP (of course it's easier to check an answer than to find one), and makes it seem that much more profoundly strange that we haven't found a way to prove this.

I have a problem with this way of thinking about NP. Just because it's "harder" to find an answer doesn't mean it can't be in P. Maybe it's just n^127 to find the answer, but n^2 to verify. I don't think it's so obvious that P!=NP as you say it is.

Also, being as that it's not proven, you wouldn't want to teach that it's "so painfully obvious that P is not equal to NP" in a classroom.

Sure, of course. I'm not out to convince anyone that P != NP here, especially not using Proof By Obviousness. I just meant to underline one of the intuitions that makes so many people believe they must be different.

Also, like so many terms in computer science, "harder" has different meanings in different subspecialities. To a complexity theorist, a problem that takes O(n^127) time to solve is not any harder than one that takes O(n) to solve -- they're both in the same hardness class. Just one of the many warpings in perspective that staring too long at P and NP can give you.

The "polynomial-time nondeterministic Turing machine" explanation has the following strengths:

1. It explains where the "N" in "NP" comes from and why.

2. For someone learning P and NP towards the end of an automata theory class, where nondeterminism and NFA-DFA transformations have already been introduced, it seems like a more elegant and meaningful development of the concept.

The nondeterministic characterization is easier to work with.

Verifying that length is less than K and verifying that length is the shortest possible route are two different questions. Can someone clarify how the travelling salesman problem is NP-Complete (i.e. how does it have a P verification?)

There's a bit of confusion here. Saying the traveling salesman problem is NP-complete means more than simply it has a P verification. Saying the traveling salesman problem is in NP implies it has a P verification. But saying that it is NP-complete implies that, in addition to being in NP, any other NP problem can be (deterministic-polynomially) translated into it.

It's mostly just a technicality of the definition. NP (and P for that matter) can only contain "decision problems," i.e., problems for which the answer must be Yes or No.

So the NP formulation of the Traveling Salesman problem is "does this graph admit a route of length less than K?" For any constant K, that's an NP-complete problem.

I thought this was the most telling line of the article: "Ultimately the first factor of performance is the maturity of the implementation."

That supports a common conviction held by fans of functional programming: if all of the years of arduous optimization that have been poured into GCC had instead been poured into (say) GHC, then Haskell would be even faster today than C is.

That is, to many people functional programming languages seem to have more potential for performance than lower-level procedural languages, since they give the compiler so much more to work with, and in the long run a compiler can optimize much better than a programmer. But so much more work has been put into the C-style compilers that it's hard to make a fair comparison. It's still hard, but this experiment seems to give some solace to the FP camp.

Haskell may potentially be compiled to within epsilon of C when using strict evaluation + mutable state + unboxed types (look at the shootout implementations; the majority are in this style), but lazy evaluation and functional data structures have a real cost.

It's not merely that the ghc developers have been insufficiently clever and diligent.

To make it even more abstruse, it's not usually results that we describe as "relativizing", but proofs and proof techniques.

For example, the proof technique we call "diagonalization" relativizes. If I prove that complexity class A is contained in complexity class B, what I'm formally proving is that a Turing machine obeying the resource constraints of class B (e.g., polynomial time) can solve any problem that can be solved by a TM obeying the resource constraints of class A (e.g., logarithmic space). If I prove this by diagonalization, then I've also proved that this inclusion holds even if the Turing machines are granted access to an "oracle" that can solve problems that the machines themselves could not solve.

Here's an example of where this gets interesting: it's been proven that there are oracles relative to which P=NP, and other oracles relative to which P != NP. This means that no relativizing proof (in particular, no proof by diagonalization) can ever resolve the P vs. NP question. (See why?)

So, oracle results don't tell us too much about the actual relationships between complexity classes, but they tell us about what kind of proofs we should and shouldn't bother trying to use to prove those relationships. It's a sign of just how hard complexity theory is that it's often considered a big achievement just to get an oracle result that tells you that the other proofs you were trying will never solve your problem.

If you're really interested, this line of research now goes beyond relativization. There's another barrier to proving complexity separations called "Natural Proofs" (http://en.wikipedia.org/wiki/Natural_proof), and more recently a third has been proposed called "Algebrization", which (far as I can tell) generalizes relativization to the case where we get access not only to oracles but to low-degree polynomial extensions of oracles. (Unfortunately, if you know what that means, you were probably already aware of the new results.)

I don't have an opinion about the Scheme -> Python switch, but I didn't find this a very compelling objection.

The author seems to assume that a CS degree is meant to teach you to write software. (E.g., "how exactly do you write software if you don't start with small working primitives (like unit tested classes, PLT scheme modules, or a PHP library script)? Isn't taking a complex problem and breaking it down into testable/reliable chunks a fundamental principal of writing software?") But who said anything about writing software? Certainly many CS grads do go on to write a fair amount of it, but it's hardly the only goal of the curriculum, especially at a place like MIT, where a CS grad is as likely to go on to a career in research as in programming.

The priorities here seem frankly bizarre; the author derides learning about "electricity whizzing through CPUs", but the principles governing how CPUs work are way more core to computer science than is unit testing, which is more of an industry best-practice than a deep principle.

Looking at it another way, a full education in CS is going to have to cover everything from processor hardware to programming techniques to theoretical foundations. Intro courses at most schools have tended to focus almost exclusively on programming. The old SICP-based intro course at MIT blended programming with a taste of theory, and the new intro course sounds like it will switch to blending programming with a taste of hardware and a crash course in real-world problems like faulty and undocumented libraries. I wouldn't say that's obviously a good move, but it's not obviously wrong, and it doesn't seem any less true to the soul of computer science.

If you're interested in wild speculation (I have no idea how Twitter's database works), here's how I interpreted Biz's explanation:

With the new system, say user Foo writes "@Bar lol me too!". Then Twitter can take Foo's follower list, join it with Bar's follower list, and send the message to everyone in the resulting list. Relational databases are very good at joins.

On the other hand, with the old system, they'd have to do a deep inspection of the record for each of Foo's followers to know if they should send the message to that follower. Relational databases are much less good at this.

But, as you and others have pointed out, if the number of users that use the "all @-replies" feature is really so small, it would be fairly inexpensive to cache the list of all of Foo's followers who use that feature, and join them in as well. I don't know why they don't do that -- maybe it adds up (like, if only 3% of users use the feature, but those users follow a lot of other users, they'll each end up in a lot of other users' caches).