Situation 3: we have understood the value of the proposed style of concurrency and think it is the right one for most situations, consequentely an according API has been created and it is strongly recommended for all usages of concurrency that fall into the pattern. This doesn't require the removal of the primitives used to build this API from the language. No linters needed, just a bit of common sense not to use the low-level API unnneeded.
In support of your point, this largely tracks way the "goto statement" shakeout has largely gone down in a number of programming languages over the decades - "available but discouraged" as in "Please do not write simple loops with if-and-goto."
Access to source inline|linked assembly is also often available. That is perhaps less of a hazard since people probably have a less easy time "convincing themselves they understand it."
But as the article points out, you can't use the goto statement as it existed before structured control flow. It's not "available but discouraged" it's just gone because it was a bad idea.
setjmp & longjmp still exist in C/C++. A great many latter day prog.langs also still allow assembly with its jump-oriented constructs. Lack of possibility is simply not the same as lack of use. Lack of use or even just lack of common use is all that is needed for the benefits (as long as the use is easily identified by either humans, compilers, or both).
std::longjmp is only defined if you're jumping "up" the call stack (the function you're jumping back into was still executing), and it would actually be safe to just abandon all the objects in your scope and containing scopes that will no longer be destroyed properly.
In all other cases it's Undefined Behaviour, all bets are off.
I'm not sure what your point is other than to elaborate constraints upon my examples. I never said it was "safe", "well defined", or even a "good idea". Only "available" and that this is an important distinction from "actually used". Even Rust has an "unsafe" construct available. I don't really think anything of what I am saying should be contentious.
Being "weakly supported" also does not imply "strictly unavailable". You can cobble together "unstructured" jumps on top of the structured exception system as in pygoto. [1] Yes, there may be logical wiggle room in some pedantic definition of "goto" or "unstructured" or "available" exactly matching what you think this article said. It is true enough for the general question of "code clarity guidance purposes" which is, in my view, the general topic both of the article and these comments. One can still meaningfully say, "Please don't use pygoto!" even though "goto" is not, exactly, "in" the language.
Many aspects of what is clear/confusing/easy/hard are not what is "strictly possible", but what a prog.lang or even its social community shepherd users to do, often through stdlibs. [2] Exceptions were originally sold as "structured error handling". Many people seem to dislike exceptions for their remaining non-local properties. In a lot of ways exceptions are often "optional", but if a stdlib uses them heavily then opting out becomes as impractical as replacing the stdlb.
In exchange for giving up go-to we gained a lot because it turns out that programme structure enables us to successfully write more complicated programs. There's no practical difference between what you claim is "weakly supported" (if I carefully read the assembly for each build to find the correct values to plug in for a jump instruction I can use in-line assembler from a language like Rust to "go-to" somewhere else) and "strictly unavailable".
In the case of C++ and Rust they'll both tell you what you want to do is "Undefined behaviour" although Rust also ropes it off with unsafe. Doesn't work? Too bad.
Keeping the flexibility you desire so much would have been tremendously expensive. It would mean all the programs real people write are slower and harder to understand just to support some unspecified "alternative" and so, as we see, nobody did it.
And that's how this insight applies to concurrency. Modern languages are much superior for being able to assume go-to isn't a thing, and the argument says they'd be much superior for being able to assume Go statement style concurrency isn't a thing.
The difference is that (aside from you apparently) it has been generally accepted that go-to was a bad idea, while prohibiting something like the go statement or Rust's thread::spawn are not yet seen in the same light. The contention is that if we did that we'd be able to understand more sophisticated concurrent programs. Seems reasonable to me.
> flexibility you desire so much .. (aside from you apparently)
Not only did I never advocate for broad usage, I explicitly said I didn't (as did @_ph_) and never expressed some love for goto or even "keeping it" whatever that means given it's often practically "possible". I neither use nor advocate Go, nor Rust, nor similar concurrency primitives in Nim, nor goto. Sheesh. Weird flaming attacks on strawmen there. You should probably just be ignored. Oh well.
>no practical difference
One practical difference is that safer/easier higher-level constructs can be written more easily in terms of lower-level - which @_ph_ was specifically suggesting a few posts up subthread. I hear the Rust ecosystem has a lot of usage of "unsafe".
Making "possible" things "too hard" can force core compiler teams into provide 1-size-fits-all solutions. All approaches have trade-offs. Google|MSFT|.. can fund compiler teams to do whatever. You may hire a bunch of kids out of college and don't want them to have too much power because they have yet to learn the responsibility Spider-Man picked up as a Teen Scientist.
>much superior for being able to assume go-to isn't
Compilers can usually analyze what the program is doing and observe "no go-to: do the better thing". They can even encourage users by saying "optimization only works if not doing PDQ" (like only tail calls get turned into iterations). APIs can always have caveats like "Incompatible with XYZ". If usage is rare and easy to identify (as I said) the situation is essentially the same from a clarity/expense of developing complex systems point of view, but I am just repeating myself to seemingly deaf ears.
To be more concrete, people don't use either setjmp or goto much in C++ because other higher level things are easier/they are taught not to. There are usually many unused dark corners of any prog.lang more than a few years old. They have a cost, but it isn't as monumental as you make it out to be and so removing it does not produce the major savings you imagine (relative to discouraging/making rare).
>Rust's thread::spawn not
So don't call it. Write a structured concurrency lib. If there aren't already 5 crates.
>are not yet seen in the same light
Seen by who? You? You speak for all systems programmers & professors now? The article author does? Or did 4 years ago? I would grant there are education problems on this and many other topics. Maybe you calibrate your "not seen" against a pool of underexposed people?
>be able to understand more sophisticated concurrent
Just use a higher-level API? I don't see a strong argument (either by you or in the article) that only prog.language support can "sell" this approach/keep things clean. @_ph_ suggested a lib. All I did was say history supported his side point and you started jumping all over me.
There are many abstractions you need for more sophisticated/complex systems..not just this. "Must..get..compiler..team..onboard!" jitters are usually misplaced. If one feels that way often about syntax needing to follow semantics, then I suggest Lisp or Racket or Nim or something where you don't have to wait for compiler teams.
It may be for "other Python reasons" that Smith claims you cannot get "safety" without prog.lang support, but Python is really just one lang and famously not good at safety in general. libdill [1] did this pretty manually as a C library (with, sure, probably a lot of other "C problems") back in 2016 - years before this article (which yes, mentions it in a snarky footnote). Pretty sure the ideas are all ancient. The same footnote mentions some Rust crates you could evaluate if that's what you like. Core concept systematization may well date back to Henry Ford's assembly lines (or earlier, but probably not, say, Aristotle!).
So, ok..Maybe it's catching on now in more application areas/stdlibs/a prog.lang or 3. I'd even agree that's probably a good thing. I never thought otherwise. I don't think I said anything to suggest that I did.
-----------------------
Look - I'm glad "fork-join, abstraction, and clean nesting" seem to have new fans in Smith, in @tialaramex, and in the world. To many, it may seem like "news". To many others, it is not "news" at all. Nor is the challenge of getting users to use safer, higher-level constructs when "cheats" are "always kinda possible" or maybe more sadly "what some popular intro/demo code uses". Bad examples propagating are a problem - not only in software even. There are surely interesting questions around the topic. I never challenged that, either.
> Not only did I never advocate for broad usage, I explicitly said I didn't
It's not about "broad usage". You said it should be "available" and it isn't in any practical sense.
> removing it does not produce the major savings you imagine (relative to discouraging/making rare).
For setjmp/longjmp C++ already took the major savings you're thinking they won't get. Hence the Undefined Behaviour. If they wanted to not have undefined behaviour for go-to they would need to substantially re-engineer the language and that would have terrible performance. So that's not what they did.
The C++ goto keyword is completely de-fanged. Suppose you've got a loop in a scope with a complicated, expensive to destroy object. A 1960s go-to person would anticipate that if they use goto to leave the loop they are thereby skipping the expensive destruction - however in C++ it doesn't do that, the compiler performs the expensive operation to clean up properly because you are leaving scope.
> All I did was say history supported his side point and you started jumping all over me.
History does not support this claim for go-to as I've explained. You cannot actually use the go-to feature in modern languages, some offer a de-fanged goto keyword that does some local structured control flow change instead, many don't provide this at all. No that Python hack is just a hack, it's not actually implementing go-to in a meaningful sense, which is why their last example hits a recursion limit instead of just looping forever as go-to would.
> Just use a higher-level API?
The article's point is that this delivers much poorer results. I actually began trying to write an example showing how hard it would be to write structured control flow as a library in FLOW-MATIC and it hurt too much as I realised function calls don't exist and I need to begin again with that perspective, FLOW-MATIC has this idea of executing a range of instructions, but the instructions don't specify that range, the "caller" does, which today seems utterly backwards. I can't do it. Nobody would do it. At scale the resulting large programs would be - as the article observes - spaghetti code. Instead new languages were developed with structured control flow.
You've mentioned that you don't "advocate" Rust but I'll further assume you don't use it or maybe don't know it well. Historically Rust provided a mechanism that let you say well, see this object X exists now, and I just made a thread, I got a thread handle, and when I leave scope that thread handle gets joined, so, logically the thread can't exist longer than object X right? And thus it's safe to use X from inside the thread knowing it exists. That is inherently unsound and was abolished before Rust 1.0 (if you're not a Rust programmer the simple explanation is Rust has no problem with you just leaking the thread handle and then it won't get joined so the thread may outlive X)
Language-level structured concurrency would let you provide this sort of guarantee and take advantage of it in your programs. Rust is getting a very small taste of this, in the form of scoped threads in 1.53 as mentioned elsewhere in the HN threads, but the article's position, and I think it's made very well, is that doing this everywhere at the language level is desirable.
That is, rather than "just don't call it" the argument is "just don't provide it" as we saw with go-to. Are they correct? I think they might be. But it was frustrating to see several people in this thread insist that actually go-to isn't gone, citing things like longjmp which are far less powerful and explicitly have Undefined Behaviour in the circumstances discussed anyway.
I never said "should". Really. All I said was descriptive not normative. I explicitly gave examples where trade-offs could shake out various ways. My strongest positive claim in this whole subthread might be "lack of common use is all that is needed for the benefits (as long as the use is easily identified by either humans, compilers, or both)." I can re-qualify that as "most benefits" if you insist. You use qualified "maybe they're right" language yourself. I don't think we're even disagreeing on the only claim I made.
In the interests of clarity, when I said "available but discouraged", I meant to refer to what you are calling "completely defanged gotos". This is the real origin of our cross-talk (and maybe any you have with others). Others do not use the term "local structured control flow" for local goto - in fact I've never heard that before. You also let "defanged" do too much work. "local goto" is by no means universally considered harmless just because it not "as bad as FLOWMATIC". Local goto was very much in Dijkstra's 1968 _Goto Considered Harmful_, for example - the OG _Harmful_ article. (FWIW, Dikjstra's earlier 1965 work proposed structured concurrency notation with "parbegin .. parend" extensions to Algol 60.) I know people that hate "return" & "break" as well for being unstructured jumps. Human language is cooperative - you may need to adjust how you discuss local gotos.
My "largely tracks" history was intentionally a rough thing. I know (local) goto statements are becoming less common, but they're not quite like punch card column rules in Fortran77. D is a recent late 90s language that did. [1] V is brand new and it has it. [2] I'm sure there are many modern examples..and more still that can cobble it together like pygoto. Herculean lengths are usually not undertaken to block a pygoto/longjmp.
Since you say longjmp is "far less powerful", I suspect you have a misimpression. You can change whatever program counter is in the jump buffer and go to any (executable) address in your virtual memory that can handle the transition. One of the other things in the the jump buffer is the stack pointer. Bad ideas for most can be useful in more expert hands. E.g., people built green threads libs around longjmp before SMP/later multi-core. [3] With many stacks. Any "yield point" anywhere in a tangle of global cross-green thread control flow could become a valid jump-back target. Some early JVMs used this, too, IIRC. I don't see this as particularly less powerful than FLOWMATIC's jumps. It kind of seems more powerful to me, but power can be hard to quantify. In context, it was also defined well enough to provide cooperative multi-threading for many users. That gnu lib is still "available" to use in C++ and probably many other things in 2022. Any "modern" language with a C backend or maybe just C FFI can also probably call longjmp. Sure, maybe not desirable. I'm not really advocating anything except maybe clarity of argument/language and awareness of properties.
Sorry..I do not know/track Rust well enough to use examples from its historical twists & turns. "Retain hairy low-level, but provide nicer high-level things" is a really common layering and all this "keep the go statement" subthread was about, IMO.. There are also warnings not only errors. Such layering is probably elsewhere in Rust, but I do not know and declare no "should". Difficulty of writing "while" or function call stacks/whatever in FLOWMATIC is simply not relevant to such highly prog.lang-and-feature-specific questions, since various other kinds of abstraction power have also increased since FLOWMATIC.
> lack of common use is all that is needed for the benefits
This is already wrong though. To deliver the benefits modern languages abolished the go-to feature, it's gone as the article explains and as Dijkstra anticipated. Early on in his letter Dijkstra observes that with procedures the use of a textual index as go-to's destination won't work. I used the phrase "local structured control flow" to emphasise that the defanged goto in a language like C++, unlike this go-to idea, isn't just a jump, the compiler is going to emit all the extra work needed to alter control flow here, just as it would for break or return.
> Human language is cooperative
I have tried very hard to refer everywhere to the idea Dijkstra is talking about as go-to rather than goto for exactly this reason. The article takes pains to explain the distinction too. Co-operation is not one sided.
You cite D, which inherits the defanged goto from C but further constrains it (in D the goto can't cross an initialisation, so D's compiler needn't handle that), and then you cite V, a language whose main recent claim to fame is that it can take the game Doom written in C, transpile it to V, and then transpile that back to C, it's not a surprise to find V has lots of C warts preserved exactly as is to deliver this capability.
> cobble it together like pygoto
Pygoto is a cute hack, but it is not actually the go-to feature. Here's how pygoto "works": It takes your entire program, it snips off the lines before the one you want to "go to" and then it asks the Python interpreter to execute the resulting text with your current global state and exits with any resulting return code.
This is why, as I already mentioned, it runs into a recursion limit when you write a program which goes back on itself, if you actually have go-to then this is merely an infinite loop, but Python doesn't, so hence the recursion.
> You can change whatever program counter is in the jump buffer and go to any (executable) address in your virtual memory that can handle the transition.
The longjmp feature that's actually supported is far less powerful as I explained. What you're doing here is relying on implementation details as if they were features from the language or supplied library. There's no practical difference between this strategy and just writing inline assembler to do the jump, in a complicated program it probably just won't work for a variety of reasons and nobody is interested in helping you figure out why.
> Retain hairy low-level, but provide nicer high-level things
... provision of the "hairy low-level" in this case makes the nicer high-level things harder to use effectively. That's what the article is explaining. Provision of this capability is harmful, which gets us right back to Dijkstra's letter.
As I said, others complain about break & return. Defanged to you is not defanged to others no matter how often/consistently you say so. The statement & many of its claimed problems persist. Call the remaining problems claws not fangs. Safety/clarity is also hard to quantify. Persistence+present rarity shows abolishing was not needed for use to fade.
Portability of longjmp can mean over (at least) CPUs, libc's, OSes, assembler backends (gas vs Intel syntax). Semantic hair splitting about what difference achieves "practical", what support (by who,what,when) achieves "actually" or other squishy traits adds no insight. Sorry you're frustrated, but I don't think there's anything more to say on goto we haven't both said >=1 times. We'll just have to agree to disagree on goto.
--
On a far more practical note (that makes goto analogy even more dubious and even less worth debating), one problem with this <=1965 approach that is charming you lately is uniformity of job sizes (in time). Since the block waits for the last of N tasks to finish, utilization is capped by the slowest finisher.
This "must wait for last of N" is at least part of why this has been more popular in parallelism settings like OpenMP than in network concurrency settings. As one simple, heuristic model, the distribution of the sample max is the N-th power of the distribution function. [1] So, e.g. 1/2=F^N => (1/2)^(1/N)=F => median of max(N) is the N-th root of the median. E.g., 0.5^0.01=0.993 for N=100. The 99.3rd percentile of WAN latencies can be very long indeed (and often even minimal job chunks require facing that).
Often you won't know ahead of time who the slowpokes will be to "launch those in the background". You might test "in-data-center" and have utilization crash "cross region". It might only be "intended for in-data-center" and be misused or some in-data-center host may break. In a parallel setting, maybe some data is in L3, some in RAM, others on NVMe, others still on spinning rust or across a network + any of that other uncertainty. Disparity almost always grows with N. There are mitigations (e.g. elastic parallelism to at least keep things busy if possible, multiple pools & task migration, etc.), but duration disparity can be a real problem - things that work in tests can "perf fail" elsewhere.
The point here is that complexities & sensitivities to deployments/context exist in this replacement idea that simply did not for "goto". Technical debt acquired by "total removal" could cost more than you save by simplicity in a few ways. These things are not easy to quantify. Maybe this is all obvious. But maybe not. I did not see such mentioned in any of the 3 discussions so far (in search for text near "last" "wait" "long" "slow" "idle" or "util", anyway) in over 300 comments so far.
> one problem with this <=1965 approach that is charming you lately is uniformity of job sizes
Er, no. I can imagine Dijkstra may have faced similarly confused people, who just couldn't understand how (for example) a loop feature should replace their use of a go-to for conditionals. Dijkstra does not propose a single new language feature to replace go-to one-for-one, and indeed makes clear in his letter that although this has been proven technically possible it's obviously the Wrong Thing™, likewise the replacement for go statements isn't a single "must wait for last of N" feature.
Just as modern programs typically have loop constructs, and conditional constructs, and procedure constructs even though a go-to statement could have expressed all these and more, the idea is that you could have several different concurrency mechanisms, each better suited for its purpose than the "go statement" and similar features today.
I think we'd want semi-short-cutting concurrency for example. Suppose there are computations F1, F2, F3, F4. In serial programs we can imagine writing f1() && f2() && f3() && f4() to have each computation happen only if the previous computation was unsatisfactory in some way because we have short-cutting && operator. But as a concurrency primitive you might like to have a way to perform F1, F2, F3 and F4 so that perhaps all four might happen however once you have a satisfactory answer from any of them the computation overall stops. Unlike the short cutting && operator this would not promise that F4 doesn't run if the others succeed but it would promise not to expend further computation after the answer is discovered.
Now, of course you can build this from the more powerful "go statement" primitive, but that's not the point here, you could build the short-cutting && operator from the go-to primitive too. The point is that having a sufficient arsenal of such structures provides clearer concurrent programming which empowers us to write more complex concurrent programs that are correct.
>> one problem {i.e. "insufficiency of arsenal" in your lingo}
> Erm, no [..needless snark, insinuates I'm confused.. proceeds to largely restate Smith's article and himself, though a little better this time; Good job! but does not solve the problem if a user wants even "most" of the N answers..]
I suppose I could have instead said "exist in this one replacement idea[..]could cost more than you save either in other constructs or in mitigations" or something similarly guarded/defensive. Seems you probably would have picked
yet another strawman or willfully obtuse reading to attack rather than argue in good faith.
If you do not have positive "arsenal addition" suggestions to solve the essence of the problem I mentioned -- which I am pretty sure many care about -- whatever gripe you might have with how I describe it then I don't think we have anything more to say to each other here.
> If you do not have positive "arsenal addition" suggestions to solve the essence of the problem I mentioned [...]
To the extent that you're still looking for the drop-in replacement you're going to of course be disappointed, this is why that Python hack doesn't do what you thought it did. Composition is what makes the higher level constructs worth having despite the fact they aren't drop in replacements and don't deliver the raw power of the simpler idea.
I encourage you to think about such higher level, composable, structures. You can imagine them as a library calling your more powerful primitive if you like, but don't be surprised if that doesn't turn out to be the reality.
Your rate of misreading & over-attribution with the seeming intent to inflame is high.
Trying to be enough of a troll to rollback a bit the perception Rust has a
welcoming community?
If you/anyone is curious about a multi-prog.lang project starting from structure
and later building arsenals in the senses here, there are worse places to start
than the 25+ year history of https://en.wikipedia.org/wiki/OpenMP . There may
be records to be found of board meetings/rationales discussing trade-offs &
limitations.
> You can cobble together "unstructured" jumps on top of the structured exception system as in pygoto.
As I explained, all pygoto is actually doing is recursively invoking Python's interpreter. It's a clever hack, but you can't actually use this as go-to, which is why the trivial loop doesn't do what you'd expect.
> Trying to be enough of a troll to rollback a bit the perception Rust has a welcoming community?
I'm sure Rust's "welcoming community" can point you to more agreeable people if that's what you're looking for.
I am an old man, and this sort of thing makes me feel older every day. Yes I am aware of OpenMP, but I learned this stuff with MPI since OpenMP didn't exist until I was a graduate.
Did you notice, in reading that 25+ year history (it would be about 30 years for MPI) a gradual trend? That things everybody was sure were just too abstract and high level to be useful in 1995 are today mainstream and of course modern versions of these APIs must offer them? Complexity is inevitable, and so ease of composition becomes more important and the structured concurrent primitives are easier to compose.
Just the other week, I was eating lunch with some people from the floor above, who look after our supercomputers and they were moaning about a high value user who has sloppily left some old spin-when-done code in the version they're actually running. He's likely bringing in enough revenue to justify the supercomputer on his own, but it'd be nice if his code would correctly detect "I'm done, exit" and let other jobs run without manual intervention. He hasn't done this on purpose because he's evil, he just did what was easiest in the 1990s tools he's using. Structured concurrency could have made it easier for his program to have the same performance and exit cleanly with no effort.
Situation 3: we have understood the value of the proposed style of concurrency and think it is the right one for most situations, consequentely an according API has been created and it is strongly recommended for all usages of concurrency that fall into the pattern. This doesn't require the removal of the primitives used to build this API from the language. No linters needed, just a bit of common sense not to use the low-level API unnneeded.