Why is there so much lousy software in the world?

We're literally surrounded by crap software. Pension funds stumble upon decades-old batch scripts with erroneous assumptions . More than a hundred million social security numbers and other sensitive data are leaked from credit institutions . And that's not to mention the heap of buggy and annoying software created by both small vendors and large corporations.



Such incompetence would never have gotten away with other engineering disciplines. Under no circumstances would we accept bridges that have as many bugs as the average software system. Why is the software design situation so dire? Why are there so many fig programs in the world?



To understand the reason for this, we first need to understand how the skill of the developer relates to the usefulness of the software that he creates to solve a specific problem.





In the first category, we see "developers" who are essentially incompetent. You can find a wide range of skill proficiency in this category, but everyone in it is similar in that their end product is functionally useless. Computers are ruthless in this regard - if you don't know how to get your application to compile, you might as well not write it at all. If you don't know how to create the basic functionality of your application, then absolutely no one will use it. Below a certain skill threshold, you cannot create any usable software. Most "developers" in this category experiment with code in their spare time, and rarely create professional software.



However, once we cross the threshold, the situation changes dramatically. At the bottom of the graph are the developers who are able to create minimum viable software systems. And if these developers improve, then the usefulness of their software also grows rapidly. An application that "worked" but was slow, buggy, unsafe and confusing is gradually becoming less buggy, more performant, safer, and easier to understand. Unlike the other two categories, as developers in this category develop skills, the software they create becomes significantly better.



Finally, when developers reach a certain skill threshold, they cross it.

and fall into the third category. In a category where everyone has reached such a high level of competence (in relation to the task they are solving) that further personal growth will have a minimal effect on the final product. For example, any randomly selected engineer from Google can create a CRUD application as well as Jeff Dean .






In an ideal world, the only developers in the first and second categories would be students or young professionals. And all professional software systems would be mainly created by developers from the third category. Developers who have perfectly mastered all the skills necessary to solve their problem and create solutions that are very close to the theoretical ideal. In such an amazing world, all the software around us would be at approximately the same high level of quality, would work exactly as expected, with optimal performance and without security holes. In this world, society would react to all software with joy, not annoyance.



However, there are two problems between us and this utopia.



First, the number of developers in the third category is extremely small compared to the second category. Programming is "easy to learn but hard to master." Millions of people can write a functioning script, but very few have mastered the art of software design. In addition, there are no obstacles to getting into the software development industry - there is no analogue for programmers to the American Medical Association or the International Bar Association. It is not surprising that there are many more people with initial and intermediate competence in it compared to the number of competent specialists.



Second, the demand for developers is incredible. Software developers have the opportunity to make tremendous contributions to virtually any industry. Compared to more niche professions like astronomy, where opportunities are severely limited, software development is an area that is sorely lacking in talent. That is, the main problem is finding talented software developers, not finding suitable jobs for them.



If you combine these two problems, it turns out that most companies that want to hire expert developers do not have the opportunity to do so. There are not enough good people to hire, and those on the market often receive incomparably better deals from FANG-level companies or skyrocketing startups.



Therefore, all other companies make the most optimal choice in such a situation. They hire developers from the second category. Developers who can be generally called “good enough”. Their applications have bugs, security vulnerabilities, and they cannot handle high loads. But at least these people are able to create something "working". Something more useful than the status quo. Something that can be rolled out to production with minimal critical analysis.






It can be misled that this situation is natural for the whole world and all professionals. But actually it is not.



There are many positions that require serious training, but fall into the category of "easy to learn". These are professions like a taxi driver, a construction worker or a bartender. In such professions there is a large proportion of workers who have reached the category of high competence, and further improvement of skills practically does not bring any benefits.



There are also many professions in which opportunities are so limited that employers prefer to hire only excellent specialists. For example, this applies to pianists. Amateur pianists can be found in many families, but they will never play in a concert hall, as the number of talented pianists far exceeds the number of planned concerts.



Finally, there are professions that are difficult to master and with many opportunities, but have regulatory barriers to entry. For example, medical practice. There were many people in the pre-Obamacare world who could not afford health insurance and would happily choose "at least some" healthcare instead of none at all (is it worth doing this is another difficult question)... However, due to strict rules, only those who have proven their high competence can provide medical services. The situation is similar in many engineering disciplines subject to strong regulatory scrutiny, such as the construction of bridges and skyscrapers, and the manufacture of medical equipment.






Software development has become a curious intersection of all of the above. It's easy to learn, so there are legions of “good enough” developers around the world. Developers who create quite useful software solutions, but full of bugs and security vulnerabilities. At the same time, this profession is difficult to master at a high level, so it is much more difficult to find developers who can avoid such traps.



The software development industry is so rich in opportunities that most companies find it almost impossible to hire good professionals. It is so rich in opportunities that despite having legions of rookie developers around the world, there is work for each and every one of them.



Finally, in software development, there is no input control. Anyone can take a coding training program, take a few online coding courses, and start offering their services on Upwork the next day. And the product of their labor can be immediately rolled out into production without any regulatory control.



Putting it all together, you can understand why there is so much lame software in the world. Software is eating up the world, accompanied by bugs and security holes.






: , , . , .



, , . CEO, «» — , .



image



All Articles