[FoRK] [fonc] Software Crisis (was Re: Final STEP progress report abandoned?)
J. Andrew Rogers
andrew at jarbox.org
Sat Sep 14 19:11:36 PDT 2013
On Sep 9, 2013, at 11:02 AM, Ken Meltsner <meltsner at alum.mit.edu> wrote:
> From the customer's point of view, the high productivity was less important
> than the rarity of individuals with those skills. Sometimes, it's easier
> to go for the popular/mediocre instead of the highly productive and
> esoteric -- companies are usually more comfortable with programmers as
> fungible commodities instead of rare, special snowflakes.
This is exactly the problem though. Popular/mediocre is able to solve fewer and fewer problems that people will pay you to solve. The proliferation of open source and inexpensive software that solves the "easy" stuff raises the talent bar for software that is worth developing. The software fruit is decreasingly low-hanging and this is shutting a lot of mediocre software developers out of the job market.
Companies still view snowflakes as commodities, it is just that the market is becoming more obviously stratified as the difference in value for developers in each strata becomes more extreme.
One brilliant computer scientist cannot be meaningfully replaced by ten mediocre computer scientists for a growing fraction of software development projects. The barrier to entry in terms of skill and ability is slowly becoming higher every year but humans have not become materially more competent at designing complex software. We have better tools but that primarily improves the productivity of people that have the ability rather than elevating the ability of people that have little. At the same time, the demand for new software is not slowing in the slightest. Consequently, this trend toward stockpiling snowflakes makes a great deal of sense for companies that are strongly dependent on their ability to develop sophisticated software. And more companies are recognizing that their future success depends on their ability to design sophisticated software as more of the world becomes increasingly data-driven.
> Given the sort of programming I've seen in the wild, I think the talent
> pool is exceedingly shallow when it comes to great programmers, but that
> most employers manage to get by with less-awesome individuals, in part by
> dumbing down the problems, taking on long-term maintenance headaches, etc.
> In fact, I suspect many IT departments have never dealt with a really good
> system as a result -- mediocre and buggy is the norm.
This is what makes it so easy to replace their existing software developers with open source software. We are in a market where "less-awesome" is becoming "no utility" to a first approximation. And companies are starting to recognize this in a big way.
It is the Mythical Man-Month taken to its logical conclusion in a straightforward economic calculus.
More information about the FoRK