Very cool. I own the originals and the 4th but sadly I have to admit that even though I am a reasonably accomplished developer with a couple of CS degrees I have a hard time understanding this work and have not yet really read it. I feel like I am publicly admitting failure just posting this. I still have time to make sure these are not just bookshelf decorations though.
The 1st Chapter is one of the best mathematical discussions I've ever read, bar none. But hardcore math isn't needed for most of us programmers. Knuth knows this.
I suggest skipping around on an "as needed" basis.
Try Volume 1, Section 2.2 "Linear Lists", as a starting point. I promise you its an easier read than you expect. Despite being relatively easy material, you will probably learn something about arrays with just maybe ~3 or ~4 pages worth of reading material.
Start at the start of Section 2.2. Maybe work your way backwards to Section 2.1 as you realize its not as hard as you think (so you start at the "proper" starting point). And then go on from there.
Its really basic stuff. But hitting "rock bottom" on the subject of arrays, stacks, queues, and such is fascinating.
He wrote his "Concrete Mathematics" book specifically to address your first point. I admit its the only book of his I read cover to cover and it wrecked me.
I agree on the section on linear lists. It was really eye opening for me back when I read it the first time. It's remarkable how much useful information there is about such a seemingly simple concept.
You're not the only one. If we're honest most of us buy the series as a mark of status / pride, not out of any plan to actually read it. I mean yeah I do plan to. Probably once I've retired. But that's a long way away :-)
This is the function my copy of TAOCP serves. I'm not anywhere near knowledgeable enough to digest all of it, but when I need something and I want to understand the ground theory behind it then to Knuth I go.
My most recent use of it was to convince my team that binary search is in fact not a good interview coding question. It's surprisingly tricky and Knuth explains exactly why!
The part where he explains how a binary tree can be unbalanced, and when, was really surprising to me. Never assume that just because your accesses are random that the tree will remain balanced on average.
If you’re interested in reading them (and they are extremely readable), you should first know something (not everything) about programming and discrete mathematics. If you watched some YouTube videos on logic, sets, graphs, etc. you’ll be more than equipped. Knuth has even written a book, Concrete Mathematics, that covers these pre-requisites but it might be faster and easier to watch a Khan Academy video!
Uuuuuh. I know a lot about programming and discrete mathematics (I used to tutor both, and I've used a lot of discrete math throughout my 20 year career as a software developer) and I still struggle with the books. Maybe your definition of "something" varies from mine.
Does Khan Academy cover the math topics of Concrete Mathematics yet? Looking at their list of math courses I don't see a single discrete math course in the bunch.
Right, which was what I was getting at. The person I responded to had written:
> Knuth has even written a book, Concrete Mathematics, that covers these pre-requisites but it might be faster and easier to watch a Khan Academy video!
Which suggests KA as an alternative to CM. Which doesn't follow if it doesn't cover the same material.
Concrete Mathematics is in fact an elaboration on the chapter covering the prereq in TAOCP. It was well worth it considering how great this chapter is.
The best part of TAOCP for me is how incredibly good the exercises are. The gradual difficulty is perfect and from 20 onwards you always get some interring insight from solving them. I am especially M and HM ones which are amongst some of the best maths problems I have seen despite being in a CS book.
Honestly, despite everyone's claims of how "beautiful" the books are, I think they are rather terribly formatted.
For example, the first volume consists of 2 "chapters". Each chapter spans about 250 pages on its own. So actually not a chapter, but a section, or maybe even a book in its own right. Then, each chapter is broken into sections and subsections that flow one into the next. There's a fundamental lack of whitespace and breathing room in a book that is supposedly "beautifully formatted".
At the very beginning of the book, there is a flowchart for how you're expected to read the book. It's not linear. It's not even close to linear, even within a specific topic. So why not just format the book in the suggested reading order?
I don't agree that Knuth is a great communicator. He's verbose when he could be succinct. He's taciturn when he should expound. He's prone to flowery language with humorous barbs in his introductions (and they can be quite funny, if you are clued in enough to what he's talking about already), but then spews dense mathematical notation when he gets into details.
So yeah, there's a lot of interesting information in the book, but it's a struggle to read it. Some people will say it's supposed to be a struggle, it's meant to make you work. Why? Programming is already hard on its own. Why make the act of reading about programming hard, too? I think, for the amount of effort being put into them, they could have been written more clearly. It makes the books feel like they are designed to mark an in-crowd of mathematicians who learned programming rather than to teach people computer science.
Writing style is subjective and I find Knuth's delightful: almost every paper of his that I've read has been fun. Even in his technical papers he writes at the level an undergraduate student with decent mathematical background (or a strong high-schooler) could mostly understand, which suits me just fine, especially compared to some other academics who write only for their peers / graduate-level and higher. And in any case, all the mathematical parts of TAOCP can be safely skipped if one is not interested.
But regarding your complaint about formatting, note that the book design of the TAOCP volumes is not Knuth's doing: someone at Addison-Wesley came up with them in the mid-1960s, and the first three volumes were published with Monotype (hot-metal) typesetting. When Knuth wrote TeX in the late 70s/early 80s, his goal was simply to retain the style of the books. (The layout was fine; the publishers were trying to move to phototypesetting and the fonts were poor https://tex.stackexchange.com/questions/367058, so he needed digital fonts and wrote METAFONT to create them, and he needed TeX just to be able to use those fonts.) Perhaps the book Concrete Mathematics will be a better example for you to critique, as Knuth had more of a hand in its typesetting (though there too a book designer was involved; the article Typesetting Concrete Mathematics has some details, reprinted in the Digital Typography volume and a poor scan of its earlier publication here: https://tug.org/tugboat/tb10-1/tb23knut.pdf).
About chapter length: the initial table of contents was one book of 12 chapters; Knuth wildly underestimated the printed length of what he had written (not many people know he actually wrote the whole book, all 12 chapters, in the 1960s, amounting to about 3500 pages — since then you can say he's been updating the chapters and publishing each section when he thinks it's ready), so the plan changed to seven volumes with the same chapters. And the chapter sections' lengths have themselves grown with the growth of the subject; as you can see, the current volume 4B is just a tiny fraction of the projected Chapter 7: covers 7.2.2.1 and 7.2.2.2.
Also, about the flowchart for how to read the books: it's partly a joke about flowcharts, but the suggested reading order is very much linear: https://i.stack.imgur.com/UCsOx.png — in words, it's just something like "read the chapters in order, skipping starred sections on the first reading. Also, feel free to skip boring sections, and skim math when it gets difficult" (he addresses the mathematics in the preface on p. viii).
That flowchart needs a HEFTY dose of https://www.drakonhub.com/, and I ALMOST had finished remaking it in it just now, but a terrible mistake on my part wiped my progress; thanks, Android. :(
Except everyone who talks about how to read the books successfully--including the books themselves--talk about jumping around in them in a way that requires a lot of skimming.
I don't find the text in any other book I own to be insufficient, or even noticeably that different from Knuth's books (with the obvious exception of some fly-by-night print-on-demand stuff purchased off Amazon in recent years). It's an assertion in want of proof. TeX might be the greatest example of Yak Shaving in human history: Knuth spent at least 30 years between Volumes 3 and 4 to work on TeX because of dissatisfaction with layout in V3.
This is what I'm talking about. Notice the use of colored blocks and whitespace to break up the flow of the page, to callout different asides and section headers.
> everyone who talks about how to read the books successfully
I will be a counterexample: I read all three cover to cover, and seriously attempted every single exercise (although in some cases, "attempted" just meant "actually understand what he's asking, which in itself sometimes took me several days). I didn't skim or jump around at all.
I like TAoCP's typography better in a subjective aesthetic sense. But when I'm actually trying to read, understand, and work-from workbooks/exercises, I find the Linear Algebra example a lot better.
> Why make the act of reading about programming hard
These books are not really programming books, they are math books. There are oodles of "programming" books out there that teach algorithms, like how to implement a quicksort and why X algorithm is better than Y for problem Z. Knuth's books are more like a graduate level algorithms or discrete math course.
>then spews dense mathematical notation when he gets into details
Because the Art of Computer Programming isn't really about programming per se, it's about the theory and analysis of programming IMHO. I'd say O'Reilly books for more famous for learning programming.
I vaguely recall an anecdote being told here on HN where Donald Knuth met Steve Jobs. Steve Jobs said something along the lines of "I admire you very much, I have read all of your books!", to which Donald Knuth replied something to the effect of "I don't think so." (more politely, though, I presume).
Which is to say I don't think many people have actually worked their way all through them. Neither have I, there's no shame in that.
And, as you point out, it's not too late to give it a try.
https://www.youtube.com/watch?v=DmbGs290qeM -- Dr. Knuth sets the record straight at a talk being given by Randall Munroe (of xkcd fame). Spoiler alert: "I only met him a couple of times [...] and in each case I was impressed by him probably more than he was impressed by me."
the impression i have--i'm not sure where i got it from, could even be from the first book itself--is the art of computer programming's mission is to be a useful resource if, a thousand years from now, computer science finds itself starting from scratch. contemporaries can learn from it, but by even just buying copies you increase the odds that the books can be recovered and used in that hypothetical situation.
Same here. I was toward the end of an associate's degree in programming and finding Knuth Vol 1 in the library stacks felt like discovering there was a whole continent of intellectual effort where I'd only seen a tiny neighborhood. It helped motivate me to go get a BSCS. But 40 years later, I've still only dipped into it from time to time (I did buy my own copy :))
My dad gave me them as a (fantastic) gift. They're tough. I haven't sat down and really gave them the old college try yet, but I want to at some point. I do feel a bit like a phony having them on my shelf without reading them yet though. They are brutal though.
I always viewed them as more of a reference where I could read up how to find and implement somewhat specialized algorithms. I really lack the discipline/interest to read those from front to back, so you are not alone. Incredible books though!
I've felt exactly like this about NKS. I have tried many times to understand it, but I come up empty. Half the time I think "well, maybe IT IS just nonsense?" and half the time I think I'm probably just not mentally sharp enough to get it. But I try every couple years or so, but the activity has yielded little more than being a paperweight in my house and a quiet exercise in looking at pretty pictures for me.
Honestly I wouldn't put NKS in the same level as TAoCP. NKS is way more experimental, TAoCP is more of a survey of the entire field. NKS is mostly analyzing the results of his experiments, TAoCP is more about the complexity analysis and proofs of the various algorithms.
Also, there couldn't be a bigger contrast between Stephen Wolfram and Donald Knuth. Knuth rightly should have a big ego, but he is a very humble, generous man. Wolfram seems to have a chip on his shoulder, fights with other academics, seems to want to be given credit for being "first!" on things.
I love Mathematica, but Wolfram himself is a controversial figure who often gets in the way of his own work.
