To Reduce Tech Debt, Eliminate Dependencies (and Refactoring)
If you want to eliminate technology debt, then stop using open source libraries and frameworks, advised Robert Lefkowitz, the recently retired Chief Architect of eyewear provider Warby Parker.
“If you don’t want tech debt, avoid using libraries or frameworks,” Lefkowitz said, speaking during his popular annual talk at the O’Reilly Software Architecture conference, held last week in New York. He also advised writing code in such a way that it does not need to be refactored later.
Lefkowitz admits that his view could be an unpopular one in open source communities, for whom sharing packages is a practice upon which they are built. In many cases, however, the sharing of packages leads to more difficulty than it is worth, he argued.
In a recent survey of its users, Appian found that 58 percent of organizations said: “that technical debt was a barrier to developing the applications needed to take advantage of business opportunities.”
Wiki creator Ward Cunningham coined the term “technical debt” in 1991. Just like someone can run up a financial debt by purchasing goods and services that need to be paid off at some future time, so too can a quick fix or hasty implementation requires resources to fix at some later time, robbing energy from the pursuit of adding new features or capabilities.
Often though, Lefkowitz countered, the issues come not from within the code itself, but rather are introduced through the use of third-party libraries.
Dependencies upon Dependencies
As an example, Lefkowitz pointed to React.js, a popular library for building front-end web components. Facebook designed React to simplify the design of a complicated web and mobile front end for a billion users. Most companies, however, don’t have projects that are quite so large. In such cases, using React would slow development down.
As an experiment, Lefkowitz removed all the dependencies from a sample iOS application, which was 80,000 lines long, including external libraries such as Alamofire, a Swift library for HTTP made up of 80,000 lines of code. The missing functionality could instead be sourced to the operating system itself (iOS already understands HTTP) or recoded from scratch. This way, he was able to whittle the program down to 35,000 lines of code. That is 45,000 fewer lines of code to be responsible for.
Even if the code is imported, if that code gets updated by the owner, then that code needs to be updated in the programs that use it as well. “The tech debt that you incur, is all the code that you borrow, and all the code they borrowed from, and so on,” he said. An application that relies on 300 dependencies, and each one of those dependencies is updated once a year, this means the application must be rebuilt nearly every day of the year.
Who do people use frameworks when they know it complicates the deployment and upkeep? A programmer may initially use a library to cover some functionality to difficult to implement, or perhaps work with an unfamiliar platform.
While the idea of sharing code so each developer doesn’t have to reinvent the wheel so to speak may seem appealing, Lefkowitz argued this only works for very large projects — such as those with 5 million lines of code or more (i.e. a database).
In these cases, it may take longer to build the functionality from scratch, but in the long run, it will actually consume fewer programmer hours. Take the LeftPad debacle from 2016, which involved a package for filling out a line of code with blank spaces that were removed from NPM repository. This move broke hundreds of thousands of programs that relied on it. Yet, this is functionality that could have been easily created by scratch.
“You have to master the tools that you have,” he advised. Pick the technologies you can solve the most problems with, even if they are not the best for solving every problem. “If you know how to use JavaScript, a lot of things that React supposedly does for you, are actually built into JavaScript.”
Refactoring
For Lefkowitz, the goal should be to write code that does not need refactoring, ideally. He pointed to the example of QMail, which was written by Daniel J. Bernstein, who offered $500 in 1997 for anybody who found a bug in the program. later, he increased the amount to $1,000, as no takers came forward. No one found a bug until 2005.
Lefkowitz pointed to his own work at Morgan Stanley as well. In the early 1980s, he developed a database system for ultra-high speed transactions. He left the company in the early 1980s and got a call back in the late 1990s, he said. The company was still using the database system without issue but it needed to be decommissioned because the platform it ran on was not Y2K compliant.
Though sometimes refactoring is inevitable. One piece of widely-used software currently going through refactoring is Kubernetes, which was originally written in Java, pointed out VMware senior developer advocate Kris Nova during a separate presentation at FOSDEM 2019, “The Clusterfuck Hidden Within the Kubernetes Code Base.”
Even after much of the code was rewritten in the Go programming language, many of the object-oriented concepts of Java carried over into Go, as well as some “anti-patterns” that now need to be refactored away, Nova noted. The large, primary monolithic repository for the code base can be a bottleneck with multiple contributors working on different parts simultaneously. Many of the pieces are tightly-coupled. The API server, for instance, is closely tied to the control plane, so working on the server means grappling with the control plane as well. The objects in the Kubernetes could have been implemented more easily through Go’s own built-in object definition for structs.
Speaking from ideals, Lefkowitz’s ideas can be maddening to practitioners, but those organizations that are creaking from the weight of supporting so many languages and frameworks and iterate-fast-fix-later ethos may want to consider at what point it is easier to build it from scratch, using tools with which you are most comfortable.