Burden of Technical Debt: Why it can Cost ML Developers Heavily  (2024)

A few days ago, Elon Musk, the freshly-minted owner of Twitter, tweeted apologizing for the slow performance of the mobile app. He blamed it on the software’s poorly batched remote procedure calls, tweeting – “App is doing >1000 poorly batched RPCs just to render a home timeline!”

Replying to this now-deleted tweet, Twitter engineer Eric Frohnhoefer disagreed with Musk and tweeted three reasons behind the slow app – features with little usage, time wasted over network responses, and years of accumulated tech debt. He further added – “Frankly we should probably prioritize some big rewrites to combat 10+ years of tech debt and make a call on deleting features aggressively.”

This tweet by Frohnhoefer has shone the spotlight on technical debt – a major roadblock in software development. The debt is not limited to software development but extends to domains such as machine learning – a major reason behind developmental and scaling issues.

What is technical debt?

Technical debt, which Sterling Lanier, the CEO of TurnKey, described in an article, as a boring/insomnia-inducing topic in software development – happens when teams opt for a quick and easy solution for a product to reach the market faster. More often than not, these quick fixes are not permanent resolutions, and the team ends up spending more time and resources.

The term technical debt was coined by Ward Cunningham, software developer and one of 17 authors of the Agile Manifesto, which formed the basis for the creation of Wiki. Explaining how he came up with the concept of technical debt, Cunningham said, “With borrowed money, you can do something sooner than you might otherwise, but then until you pay back that money, you’ll be paying interest. I thought borrowing money was a good idea, I thought that rushing software out the door to get some experience with it was a good idea, but that, of course, you would eventually go back and as you learned things about that software, you would repay that loan by refactoring the program to reflect your experience as you acquired it.”

Tech debt in machine learning

With the rapid progress of machine learning, an accompanying trend has also emerged – that of maintaining and further scaling them. While developing machine learning systems is now easier, faster, and cheaper, maintaining these systems has become equally difficult and expensive.

As the researchers from Google mentioned in their paper – Hidden Technical Debt in Machine Learning Systems – machine learning systems have a ‘special capacity’ for incurring technical debt. This could be attributed to the fact that these systems not only have maintenance problems of traditional code but have to also bear an additional set of challenges unique to machine learning. What’s more – these debts are often difficult to detect since they exist at the system level and not the code level. This means that typical methods that are used to play down code-level technical debts in software development are not sufficient to address machine learning challenges.

One of the most significant technical debts in machine learning is feedback loops. It happens when the output of the model is fed back as the input – this leads to a form of analysis debt that makes it difficult to predict the behaviour of the model even before its release.

Trailing close after feedback loops, in terms of how common they are, are garbage features. As the term suggests, these features don’t do much and contribute significantly to making the system bulkier – in short, absolute garbage. Very similar to garbage features are something called anti-patterns. A very small fraction of code in machine learning systems is actually doing learning or predictions – the rest of the code could be simply avoided or refactored when possible.

Lastly, dependency debt is also considered a key contributor to technical debt in both software development and machine learning systems. This often aggravates since there is no tool to detect them, leading to a build-up of large data dependency chains that are difficult to untangle. Other technical debts in machine learning include configuration debt, external impacts, and correction cascades, among others.

Strict vigilance

A lot of people believe that technical debt is not completely avoidable, given the strict deadline and delivery pressure. A necessary evil, if you will. However, its impact could certainly be reduced by ensuring best practices, coding standards, regular reviews, and others.

As Gergely Orosz, software engineer and author, wrote in his blog, a workplace that ignores technical debt gives birth to a ‘grim engineering environment’. Unfortunately, the team often gravitates towards and is rewarded for short-term solutions and hacks, which is a recipe for disaster in the long run.

Access all our open Survey & Awards Nomination forms in one place >>