There was a debate a few years ago around whether software engineering was “real engineering”. The driver behind it was the idea that for example a civil engineering team building a new bridge would spend months, years even, creating models - both physical and mathematical - and testing those models to destruction. Bridges falling down in real life are catastrophic events. Far better that time be spent up front to ensure the models fail than risk lose of life when the real thing fails. But software development doesn’t work that way. Other than with safety-critical systems, a bug in an app is unlikely to be life-threatening. It is relatively quick to fix a bug and, since the birth of the Web, it has become very easy to distribute new versions of that bug fix via upgrades. Therefore software engineers didn’t follow that “spend years designing and testing the design up front”, so aren’t real engineers.
The consensus that grew out of that debate was that there was a misunderstanding around the role of the developer from an engineering perspective. That developer writing code was wrongly being viewed as the builder of the bridge, not as the designer. By treating the code as the design and the compiler as the builder, software development makes sense as an engineering discipline. The engineer creates the design, the compiler takes that design and from it, builds a component of the app. Automated tests are then used to test that component to destruction. Even the whole app can be tested that way with end-to-end tests.
Because it is quick and easy to change the design of the software bridge, then build it afresh and test it to destruction, it is possible to release new versions frequently. The upgrade of a real bridge might take a decade. In comparison, pushing a new version of a microservice to production can take just hours and occur more than once a day. However a consequence of that ease-of-change is that it’s not just an organisation’s own software output that can be rapidly and frequently updated. The third party software tools and libraries they rely on also frequently change. And because they rapidly change, older versions go out of support much more quickly, often within a year of release.
Dealing with the constant stream of new versions of tools and libraries is a challenge for any organisation. A software development team might have 100s, even 1,000s of repositories, some of which need to be kept up to date, some of which are stuck needing to use older tools. Some will produce libraries used by other repositories. There may be legacy systems that need maintaining, but there will also be lots of systems that are in active development and the tools for those repositories need to be updated without impacting on that development flow. Keeping everything up to date can be a huge and costly undertaking.
Mantis Accounts have been established for 20 years and produce a range of accounting products, from desktop and mobile applications, through Excel plugins to an online portal. Most of the source is written in C# and .NET 8. The portal uses Angular. One Android app that the company acquired from a merge is written in Java. There are plans to rewrite it, but for now it is updated with the occasional bug fix. The Excel plugins are mainly .NET Framework-based, though some have been rewritten to use TypeScript and the newer APIs, so support the web-based version of Excel.
The development team is conducting it’s annual review of the state of the repos. A shared spreadsheet from last year is sent to the team leads, each tasked with updating it with their set repos and what they contain: apps, libraries, services or something else. The languages frameworks used, along with their versions are recorded. The process drags on for a few weeks as feature development has to take priority. Eventually, the following conclusions are reached:
Finally, Stories and Tasks can be created in Jira to record what needs doing, in what order. One team lead is assigned to coordinate the process, though it proves a frustrating job due to pushback from project and product managers who have their own pressures of on-time feature delivery to deal with and the annual upgrade just causes delays for them. But, over the course of a few months, the job gets done.
One of the key features of an IDP is that it provides a centralized information hub. In other words it provides a central location for key information that can be used to better plan upgrades:
Silverfish IDP won’t be able to do Mantis Accounts’ annual upgrade for it, and the pushback from project and product managers will still happen, but the IDP automates all of that preparation work. And because the information is constantly updated, it can become a BAU task that happens throughout the year, allowing that annual upgrade headache to be retired. Now that really is worth that $59.90 a year for each developer.