The most important thing to remember about developer compensation, especially when comparing to more "traditional" roles, is scale.
A programmer working at a large internet company may be impacting millions upon millions of people.
Building a feature that lets you collect (or save) a penny from a user every month is worth $240K a year when spread out over 2 million users.
$120M when spread out over 1 billion users.
That's really my answer to the common question at the end of the post:
"I don’t understand this at all and would love to hear a compelling theory for why programming “should” pay more than other similar fields, or why it should pay as much as fields that have much higher barriers to entry."
Scale is not the most important factor. It really is about supply and demand. If employers could get away with paying their developers 30K a year, they would (... and they do whenever possible).
The actual scale of the work certainly informs the compensation, like it might for other jobs like journalists for example, but that is only part of the story. If you need developers, you simply gotta pay them salaries the are competitive with what others pay for that particular role.
Scale is part of the demand side of the supply and demand equation.
Using dkopi's example, at a company with 2M users, it's worth hiring a developer for each "penny per user per month" problem. A company with 1B users, it's worth hiring a developer for each "0.002 pennies per user per month" problem.
Or similarly, if you look at developers who work on internal tools that save other developers time. At a startup with 40 employees, each new tools developer must save everyone else two days per year. At a large company with 20 000 developers, each new tools developer must be able to save everyone else 6 minutes per year.
As the scale gets bigger, smaller and smaller problems become worth hiring people for.
Scale IS the most important factor, because it controls the "demand" side of "supply and demand". So long as hiring someone makes the company more than they cost, the company will be willing to pay that price. Because software is so well leveraged in creating profits, it means there are nearly infinite spots where companies could make a profit by hiring a developer at current rates. The rates will continue to rise, and the supply will continue to increase, until it has hit equilibrium.
That's hardly the most important thing about developer compensation. The value a worker provides is a ceiling on their compensation, not a floor. As long as the "large internet companies" are receiving applications from more than one developer, they are unlikely to be paying $240K / year for developers who produce $240K / year of benefits. If the same people can handle building that feature for 2 million users and for 6 million users, the developers who get hired to build it for 6 million users will make 1x as much as the ones who build for 2 million users, not 3x.
How much impact does a laborer repairing a Dutch dike have? How much of that impact do they earn?
Construction workers are traditionally resources, They can be quickly replaced. With software this becomes tricky. You need to spend considerable time, amount and energy to find the correct replacement who can drive it with same efficiency. The 2 reasons why companies like Google, Facebook etc pay so much is because:
1) They want to attract the top talent, top 1%, 5% may be.
2) They want to keep people happy so that the key people working on most important features don't leave the company.
And this is where the supply side of the equation comes into play. Many people can build the road and there shouldn't be any differentiation in how it is built (assuming they have common engineering standards). Thus, the supply is almost perfectly competitive causing low wages.
However, there are infinite ways a developer can make something. 1 developer can make design a system that can save/make their company millions that most other developers would miss. The supply side of developers is more like a monopolistically competitive market. Thus, they can extract some of those savings/profits in wages.
I was thinking along that line too. In traditional more "human" field, a good lawyer/ doctor can just do that much, regardless of how good you would be.
It would be interesting to compare revenue or profit per employee for tech company, comparing to law firms and the like.
Another thing is that we believe (ie. The Mythical Man Month) in software, there is a limit of programmers per task before you just can't throw any more body into it anymore. Is this true (or still true), and does this apply to other fields of comparison?
Immigration could likely be a bigger factor than we thought. How long has developer compensation been skyrocketing? The immigration situation to the US has only been getting shittier for the past 5-7 years mostly (although it wasn't flower and pleasant before that, if you wanted to do startup or worked in the US with a job offer, it's several folds easier back then). And seeing that it takes like 5-7 years for a startup to mature, even longer for immigration "market" to catch up with reality (it takes a long time for people to decide or to immigrate in general), it wouldn't be a surprised that compensation is still lopsided for the US.
> there is a limit of programmers per task before you just can't throw any more body into it anymore
This is important. The effect is that to complete the task you have to effectively scale the team up, you can't scale horizontally, which increases demand on top talent.
It means that if a programmer is (theoretically) 10x more effective than an average programmer he may get much much more than 10x the pay.
A programmer working at a large internet company may be impacting millions upon millions of people. Building a feature that lets you collect (or save) a penny from a user every month is worth $240K a year when spread out over 2 million users. $120M when spread out over 1 billion users.
That's really my answer to the common question at the end of the post: "I don’t understand this at all and would love to hear a compelling theory for why programming “should” pay more than other similar fields, or why it should pay as much as fields that have much higher barriers to entry."