Timothy Chow has a wonderful phrase for this - he describes one of his papers (on forcing) as solving an "open expository problem": https://timothychow.net/forcing.pdf
I've been wondering at what point AI assistants are going to reduce that to a manageable level? It's unfortunately not obvious what the main bottlenecks are, though Chris and Shan might have a good sense.
It might be doable soon, you're right. But there seems to be a substantial weakness in vision-language-models where they have a bad time with anything involving screenshots, tables, schematics, or visualizations, compared to real-world photographs. (This is also, I'd guess, partially why Claude/Gemini do so badly on Pokemon screenshots without a lot of hand-engineering. Abstract pixel art in a structured UI may be a sort of worst-case scenario for whatever it is they do.) So that makes it hard to do any kind of feedback, never mind letting them try to code interactive visualization stuff autonomously.
Gwern is correct in his prior quote of how long these articles took. I think 50-200 hours is a pretty good range.
I expect AI assistants could help quite a bit with implementing the interactive diagrams, which was a significant fraction of this time. This is especially true for authors without a background in web development.
However, a huge amount of the editorial time went into other things. This article was a best case scenario for an article not written by the editors themselves. Gabriel is phenomenal and was a delight to work with. The editors didn't write any code for this article that I remember. But we still spent many tens of hours giving feedback on the text and diagrams. You can see some of this in github - e.g. https://github.com/distillpub/post--momentum/issues?q=is%3Ai...
More broadly, we struggled a lot with procedural issues. (We wrote a bit about this here: https://distill.pub/2021/distill-hiatus/ ) In retrospect, I deeply regret trying to run Distill with the expectations of a scientific journal, rather than the freedom of a blog, or wish I'd pushed back more on process. Not only did it occupy enormous amounts of time and energy, but it was just very de-energizing. I wanted to spend my time writing great articles and helping people great articles.
(I was recently reading Thompson & Klein's Abundance, and kept thinking back to my experiences with Distill.)
Huge fan of Distill here (and your personal blog).
> In retrospect, I deeply regret trying to run Distill with the expectations of a scientific journal, rather than the freedom of a blog, or wish I'd pushed back more on process. Not only did it occupy enormous amounts of time and energy, but it was just very de-energizing.
Scientific peer review pretty much always is incredibly draining, and (assuming the initial draft is worth publishing) it rarely adds more than a few percent to the quality of the article. However, newcomers are drowning in a sea of low quality SEO spam (if they bother to search & read blogs at all and don't go straight to their LLMs, which tend to regurgitate the same rubbish). The insistence on scientific peer review created a brand, which to this day allows me to blindly recommend Distill articles to people that I am training or teaching. So I, for one, am incredibly grateful that you went the extra-mile(s).
A good brief overview here from Tim Gowers (a Fields Medallist, who participated in the effort), explaining and contextualizing some of the main caveats: https://x.com/wtgowers/status/1816509803407040909
It's probably most clearly stated in the original post, but the point is you would either be able to find three people who are all friends with the other two, or you would be able to find three friends who are not friends with any of the others. The non-obviousness is due to the fact that someone may have only one friend, or someone may have two friends but those two would not be friends with each other, so I wouldn't call it blindingly obvious that there would always be a subset with one of these properties (I think the main intuition that is useful is that if you have a sparsely connected graph of friends, it's easy to find a group that aren't friends at all, and if you have a densely connected graph then you can find a group that are mutual friends. The theorem is then proving that there's no middle ground: it's either dense enough a mutual group must exist or sparse enough a non-friend group must exist.)
Your formulation is not equivalent. The actual Ramsey theory formulation is something more like...you need to color, either red or blue, all of the edges of a fully connected graph of 6 elements (there are 15 such edges). No matter which coloring you choose (there are 2^15 of these), there will always be a "triangle" between three nodes where the entire triangle is either fully red, or fully blue. If you were to instead restrict yourself to a graph with 5 elements instead of 6, it's possible[1] to color the edges so there's no triangle where all the elements are the same.
As an exercise, try repeating your same argument for 5 colors/blocks, and note that it still works, when it shouldn't.
In your version, the color of a vertex is independent of other vertices. The real problem is about entirely about connections to other vertices. The latter cannot be reduced to the former.
There's (at least) two meanings for AI: (1) Software systems based on extensions of the LLM paradigm; and (2) Software systems capable of all human cognitive tasks, and then some.
It's not yet clear what (1) has to do with (2). Maybe it turns out that LLMs or similar can do (2). And maybe not.
I can understand being skeptical about the economic value of (1). But the economic value of (2) seems obviously enormous, almost certainly far more than all value created by humanity to date.
"LLMs are no good for [use case X]" often means "I aren't very good at using LLMs for [use case X]".
With many powerful tools - violins, say, or carpentry tools - we know that it takes a long time and a lot of learning to achieve competent performance, much less virtuoso performance. Someone who spent ten hours learning the violin and concluded "Violins sound terrible" wouldn't have diagnosed a problem with violins, but with their own mastery. I certainly think current LLMs have some big intrinsic weaknesses, but also that what they are is quite subtle.
