The very specific example you chose: payroll, shows how it can be difficult to incrementally step from small to huge. As you grow from town to national, you will run into all the disadvantages without really hitting the advantages. I feel that incremental does help you move from one level to one just a few above. But only if there are enough customers at these starting levels exactly.
When developing for towns, you will have all small random subsets of the variations imposed by year after year of legal changes BUT small sales. You will have to implement niche variations in arbitrary aspects for all the towns you have to support AND you will not have the customer size on which to amortize this work. Each new customer will bring a new arbitrary set of legal aspects to be met. Each new customer may be arbitrarily difficult to support.
By the time you reach national, you will have already covered most of the historical legal quirks - but that will have been done in one kludgy manner after another - and then you will hit one more set of legal quirks at the level of national organizations (some of them will have their very own laws). You will now have a very large budget to finalize things but you will be burdened by an illogical software base.
So I agree that you will need experience and subject matter experts that have worked at the various levels. BUT, now that you have this experience you now know the degree of flexibility that is required (you know where and what needs to be variable and quirk-friendly and how far the quirks can go = "any size") as well as size-related issues (mailing, transaction, user support volume) and you can now plan for all this AS YOU restart a new development from scratch. Because at this new "master" level you need both systematic flexibility AND relience at size.
Payroll is exactly the kind of topic where "adding features" will be "fun" - I mean bewildering - while you learn, but probably economically difficult to manage, until it kills you "as you climb up"?
You will be killed by a large software project that can afford to hire out a bunch of your subject matter specialists (or hires new ones) and uses them in a "from scratch" project. If you are lucky, this large project will be from the same company but only if you are lucky.
Now. AFTER you have done the one top level project - for one country -, you will probably be in a good situation to sell service to all kinds of organizations. Because you now have a system in which you can implement ridiculous quirks without breaking everything. And if you have done the job just right, you can onboard smaller customers (towns) economically enough that they can afford your solution.
That's different from where you deploy your solution first. Sure, deploy a national-design solution first at a subset of the target employees - although that does impose more requirements still: now you need to coexist with the legacy solutions. Which would be another hard to meet handicap when developing for towns first.
> Perl 6 would have had to be a better Perl 5 and a better Python 2 to win.
Don't sell perl 6 short. I am using perl 6 for significant projects now (after a career of perl 5) - and it's fundamentally different. I describe it as perl to the power of perl.
For me, expressiveness is fundamental. And perl 6 gives me that.
Perl 6 is simply suffering from python being everywhere. And perl 5 was always easy to lampoon as "line noise". It's a stupid quip, but it leaves a mark on new programmers. You don't even need to read the course and you can already have an opinion. Stupid kills? And then perl 6 doubled down on that anyway. Then I doubled down on that ALSO and I get to use (carefully chosen) unicode symbols in my line noise :-) So there.
> For me, expressiveness is fundamental. And perl 6 gives me that.
I saw Larry Wall at one of the conferences. He talked a about Perl 6 how it was progressing and such (it was before it was renamed) and year, expressiveness what stuck out. It certainly has lots of nice features, too. But at least for me, I realized with Perl 5 I wasn't smart enough for it. I would be lured by the clever short expressions and then sometimes later look back and had no idea what I wrote.
Larry is a great person, btw. During lunch at the conference sat at his table and he was very approachable and warm. I don't remember what we talked about exactly just that it I liked how down to earth and nice he was.
> But at least for me, I realized with Perl 5 I wasn't smart enough for it.
I feel like you but I love it: I am not limited by the language. Not by perl 5 and not by perl 6: When I am willing, I can dig deeper and find more to work with. When I am willing I can try and follow presentations or books by Damian Conway, Mark Jason Dominus, etc and I can get new ideas and inspiration. I can always learn more about this fundamental tool that's at the center of what I build. The tool challenges me in a good way. It does not slow me down. It does not limit me. If anything in there is going to limit me, it's going to be my own brain.
And perl does that without tripping me. Because in perl, the intuitive way is one that's not likely to hurt you. While if you know better, you can work with the more elaborate, deeper features.
I hate it when I have to use a language that constantly limits me. It has happened. I am not always free to choose the programming language or platform. For some, it's so frustrating that I charge more. And it's still frustrating.
I have exactly zero functions that are written in both. So comparison is just intuitive:
My current projects make extensive use of numerical functions AND of regexes and grammars. Both used extensively. I am very satisfied with the performance on both aspects. It does the job. And there is no question that for the regexes and grammars, what I have goes FAR beyond what I ever dared to run in perl 5. Performance is good, including on this stuff that I feel would have been pushing perl 5.
Still, I write for expressiveness. It matters to me how fast I can write. And I am NOT satisfied with "searching through the doc". That's the main sore point for me. The doc is very good but online-first and local... eventually. So I am stuck using an online search function... which constantly falls short.
Which is covered in the very first section of the course book? Yes it has its own logic. So do lots of programming languages.
How far do we need to take "not reading the doc"? That the very first chapter is too far? People who gave up on perl because of that... really would not have survived the rest of the course anyway?
You can learn Python, PHP and (a bit less) Ruby without ever touching a book, and definitely you can skip over the first chapter if you're an experienced programmer.
There must be a reason why they made sigils more "traditional" in Perl 6, for example.
> People who gave up on perl because of that... really would not have survived the rest of the course anyway?
I didn't give up on it, I just didn't feel the need to stick with it. I switched to something else that required less of a context switch when going back, awk or jq or Python.
Now, jq is something that throws me off every time I use it. But it's a completely different processing model, whereas lists and hashes are not specific to Perl.
So, to touch on that, no contest that for a while now, you can have a well paying job only writing python. (And perhaps even never going through a formal course on it.) That has worked for many people.
> There must be a reason why they made sigils more "traditional" in Perl 6, for example.
Eh. Sigils are even more present / visible in perl 6. And other compact notation devices. All the way to making up your own unicode-based line noise when it serves. Which it does.
Yes, I didn't say that sigils went away. But for example in Perl 6 lists are @ whether you declare or access them, instead of the sigil changing depending on what you want to get out of the expressions. It says it in one of the first apocalypses even that people "found it a bit weird": https://metacpan.org/release/AUTRIJUS/Perl6-Bible-0.30/view/...
> getting the same question repeatedly may suggest something should be redesigned
Yes! There was a lesson in that and we all missed it. That was probably one of the failings of perl. It ran into a generation of people who never knew about "man pages", or couldn't read (jk - but only somewhat: for some people reading is very hard because various flavors of ADHD, dyslexia, executive disfunction, whatever) and the man page is then useless, or they go to google first and '$|++' failed (because google was raised on python).
Better marketing of the documentation would have helped.
I would say "we'll do better next time" but then perl 6... I'm not happy with perl 6 documentation. There is a lot of it - no problem there. But it insists on living online which necessitates a hosted search function. Which is always broken. And there is still no "local doc" solution.
man I hate that always online stuff. why isn't there the comprehensive man pages thing for perl 6? rhetorical question tho, I don't have much interest in perl 6.
The man pages were broken into competent sections over multiple man pages. About 260 of them on this machine (not really: there is a change-tracking man page per release recently.) The 1st man page is a very compact index to them.
After that, each section is long but very searchable.
But I can see how many people never even noticed."Man page? what's that? what for?"
But if every user is expected to read the entire manual cover-to-over before asking a question about a four-character operator, what's the point of dividing it into sections?
I know you are not serious but let's go with it anyway. Since we have to be welcoming to newbies :-) Not expected to read cover to cover. Cover to cover would be the tutorial / course work. And even then in a layered language like perl 5, the later chapters only when needed. Layered: the language and coursework is written to add one layer of the language after the previous ones. So you can start writing things that do function after just the first layer.
Expected to know it's there, navigate through it to find the operators or system variables and that you can search through the thing. There are ~260 perl man pages on this computer - not expected to read them. For damn sure expected to use them.
Do read one more section from top to bottom now and then - if the thing is the one fundamental tool in your job!
But that's what the ancestor was trying to do. Then he ran into a symbol he hadn't gotten to yet, asked about it, and got flamed for not reading the full manual!
It was not for not reading the full manual. It was for not using the manual. Somewhere between "not at all", "not competently", "not persistently". And he was pointed to a perl-specific tool which is made for searching the doc. Not the same thing?
And he/they had missed an entire category of symbols. That none of the responses pointed at - their bad on that. That is, all these symbols are described in the same manual section. And used in illustrative examples all over the place. They are not exactly a deep hidden thing.
Also, regarding "flamed". No. Not really. They were handed the same response that countless other questions were getting. Anyone frequenting these forums saw them countless times. It is quite possible that it was their first time on that forum / chat and then that the answer was shocking and traumatic. Yes to that. So that in hindsight, the standard response should have included a pointer to a "how to use the doc" doc. That would have helped. Since it was a generation was seemed unaware of the man pages.
The problem is the implicit assumption that you can throw an entire book at someone and expect them to be able to figure out where to look for each piece of information in it without them knowing beforehand what things even are. If someone hasn't ever seen a variable like `$|` before, it's not necessarily going to be obvious to them what it is, and without knowing how to classify it, they aren't going to know how to tell what chapter it's described in. You're defining the bar for someone to be able to ask a question as high enough that they spent enough time reading through everything to be able to identify it. That's going to be fine for some people, but not everyone learns the same way, and when some people learn better a different way than you, it's not because they're stupid or lazy, but because there's just a lot of variety in what works well or doesn't for people.
Of course, you aren't under any obligation to spend time helping people who you don't want to, but if a community as a whole reacts this way when someone asks a question, they're making the bet that the there are enough people who are similar enough to them to sustain things in the future. Given that this both happened years ago to the parent commenter and now again when they tell the story again, it's not really that hard to believe that this might have been common enough that a lot of people experienced it. The entire point of this thread is discussing why Perl has faltered, and your explanation in the last paragraph comes across as basically saying "kids these days..." in slightly different words. I'd argue that even if the kids loved man pages, having a condescending attitude towards them would probably still come through in other ways, and that would have had pretty much the same effect.
I am agreeing with you that there was a mismatch between the expectations on using the elaborate documentation in its various forms and the tutorials and the stellar course - and a large set of potential users. Nobody expected "the entire book" but perl 5's rise came at the time when many stopped reading man pages, and many projects stopped providing them.
I agree with you that this probably was a contributor in some people giving up perl quickly. For python or php.
Like I mentioned elsewhere, for people for whom using a book would be a barrier - perl would have been a poor choice anyway. You can't program in perl without using the man pages and books. It's a large language, with lots of features purposely made less visible to the newcomer.
In addition, many people were exposed to perl from web scripts. And it was sooo tempting to just paste in a perl script, and then want to modify it, without spending any time on learning the language. Perl makes that frustrating (and compensates with a stellar course book). I still defend perl by arguing that (in perl) there is no point in discussing what $| might mean even before having covered the basics, for example sigils. The course book is layered, and for good reason: to let you write a program in useful order, fundamentals first. The special variables come up fairly early but then again the course book had an extensive index which includes these special variables first in a symbol section, and then again in the alphabetical order for their wordy version $| or $OUTPUT_AUTOFLUSH. I'm not trying to beat you over the head with the manual. Just pointing out that the course book was throrough and intelligently written.
I'll point out that throwing a question at a forum without poking around it a little to figure out the local mores - well, still now, that will get you barked at. Lesson: Forums would do well to provide a more useful paste-in than "RTFM" - ready to go for their users. Instead of "RTFM". At least if they want to foster adoption. Does any forum do that particiularly well, that you have noticed? Most discords for example, do NOT do that well: it's possible to create stickies and they are really not visible. So people create onboarding documents which then get too long and get skipped. A problem not solved there.
You did not need to google anything. The complete documentation was right there, next to the interpreter, on your machine. Ready for scanning and reading from top to bottom; and broken in sections that were actually relevant; and ready to search in bulk if you prefered with whichever local search tool you cared for.
No need for google. (And google was run by python fans; probably saw no need to support searching for '$|++'.)
And I notice "post the predominance of offline doc". Well that's one problem right there: As of 2025, there is still nothing that beats perl 5 docs as ~260 man pages. Probably LLM-based AI is getting there, at least for people who have difficulty with text. But for the rest of us, it's VERY useful to know that there is solid (offline) doc.
Also to be fair, a modern Perl app doesn't even need to use `$|++` since the framework, even if it's raw Plack, will manage output flushing for you since you're no longer banging on raw stdout. I'd say Perl suffers from an even worse problem of legacy tutorials than PHP, but the size of PHP's userbase and thus sheer number of bad tutorials makes it worse there.
A professional medium might have been gate-kept being paying coursework. Perl was not: the super-complete documentation was right there, in the distribution; the remarkably intelligently written course book was right there on everyone's shelves and in your local public library; the in-depth books same thing; And quickly enough even free on the net; the expert-built module library was all there to use and study; the experts were giving their time freely, writing deep dive articles at the simple prompt of worthy questions in addition to columns on topics which I guess were not getting enough questions.
For a professional medium, the only lack that I can tell is from a marketing point of view: installing the distribution for example, probably did not highlight enough how extensive the documentation was.
Perl was not and still is not suitable for people who refuse to read the manual or tutorial.
Perl had and still has truly AMAZING documentation (and far better in perl 5 than Perl 6 - where the documentation's search function is regularly broken for example). And out of this world bring-up tutorials. But if the bar is at "I want to be up to speed after reading a short booklet" - well, that's not gonna work. For me I have programmed in perl for decades and I still get the occasional deep plunge in a feature area that I never used before... And the various docs and books still come through.
On top of that, Perl 5 had an amazing community ready to help - but understandably annoyed with constantly re-quoting the AMAZING documentation. It could have gone either way, it just went the way it did: experts present and ready to answer. But there too - if you expect the community to constantly answer the most basic questions, well, Perl was not for you.
As a tradeoff perhaps, that expert community also spent a lot of effort on the AMAZING module library.
Perl 5's thorough documentation, plus splendid course book, plus all the books, plus CPAN, PLUS the expert help producing fantastic rabbit holes to learn more about your PROFESSIONAL MEDIUM. THAT was the reason for choosing perl.
If anything, newcomers don't go to the documentation because they are not aware that documentation can be THAT good. That was, still is, a marketing error.
When developing for towns, you will have all small random subsets of the variations imposed by year after year of legal changes BUT small sales. You will have to implement niche variations in arbitrary aspects for all the towns you have to support AND you will not have the customer size on which to amortize this work. Each new customer will bring a new arbitrary set of legal aspects to be met. Each new customer may be arbitrarily difficult to support.
By the time you reach national, you will have already covered most of the historical legal quirks - but that will have been done in one kludgy manner after another - and then you will hit one more set of legal quirks at the level of national organizations (some of them will have their very own laws). You will now have a very large budget to finalize things but you will be burdened by an illogical software base.
So I agree that you will need experience and subject matter experts that have worked at the various levels. BUT, now that you have this experience you now know the degree of flexibility that is required (you know where and what needs to be variable and quirk-friendly and how far the quirks can go = "any size") as well as size-related issues (mailing, transaction, user support volume) and you can now plan for all this AS YOU restart a new development from scratch. Because at this new "master" level you need both systematic flexibility AND relience at size.
Payroll is exactly the kind of topic where "adding features" will be "fun" - I mean bewildering - while you learn, but probably economically difficult to manage, until it kills you "as you climb up"?
You will be killed by a large software project that can afford to hire out a bunch of your subject matter specialists (or hires new ones) and uses them in a "from scratch" project. If you are lucky, this large project will be from the same company but only if you are lucky.
Now. AFTER you have done the one top level project - for one country -, you will probably be in a good situation to sell service to all kinds of organizations. Because you now have a system in which you can implement ridiculous quirks without breaking everything. And if you have done the job just right, you can onboard smaller customers (towns) economically enough that they can afford your solution.
That's different from where you deploy your solution first. Sure, deploy a national-design solution first at a subset of the target employees - although that does impose more requirements still: now you need to coexist with the legacy solutions. Which would be another hard to meet handicap when developing for towns first.
reply