Enterprise Devops gets “stuck in the middle”, but legacy tech is not the reason
I work with many large enterprises. Most of them made a start with DevOps at some point in the last few years. Most of those have not yet progressed to a significant level of maturity. It could be tempting to regard old technology as a significant cause of this problem, but is that really the case?
It has been interesting observing this trend since the publication of the 2021 edition of Puppet’s “State of DevOps Report”. This was the first major annual report of its kind which explored in depth an issue which I have frequent observed in my own work as an enterprise-focused software product manager.
The report used the phrase “stuck in the middle” to summarise one of its key observations: that many large, established enterprises are stalling when attempting to grow their DevOps footprint. After promising, but limited-scale starts, it is all-too-common for the early momentum to be lost.
I’ve had many exploratory conversations on this topic. Typically, the experiences of the people I talk with mirror the findings of that 2021 Puppet State of DevOps Report. Pockets of transformation have indeed occurred, and there is often a keen but localized enthusiasm to push DevOps wider.
In defining levels of evolution, the authors identified four key “blockers to DevOps”, and measured their prevalence in each organisation:
- Organisational resistance to change
- Legacy architecture
- Shortage of skills
- Lack of automation
The majority of companies measured had achieved a state of “mid-evolution”. The report defines “high-evolution” organisations as those which had completely overcome organisational resistance to change, and a lack of automation, to reach a state where only legacy architecture and shortage of skills remained as impediments.
In a 2017 article in CIO.com, HSBC’s then-head of IT operations discussed his own shift from idealism to pragmatism:
“Three years ago, I believed we’d be able to retire all our legacy and move everything to the cloud. But it quickly became obvious that in the short to medium term we were going to need to keep our legacy systems running while also investing in the cloud and digital technology.”
Banks are a really good example of the kind of enterprise which has needed to transform its practices and offerings to be more digital. These changes reflect in the experiences we have when we interact with them.
For example, consider the process of transferring money from one retail bank account to a similar account at a different bank. Before internet and mobile banking became widespread, this would generally involve a face-to-face or telephone conversation with a human customer service agent. The agent would be using a dedicated application, running on a physical client-server stack. The application would set up back-end transactions which would typically run some time later (perhaps overnight) as part of a periodic batch process.
Today’s modern banking experience changes the technology interaction. The customer is typically directly linked to the technology, without a human intermediary. The technology stack driving that interaction has changed significantly, away from a single client-server application to a multi-channel set of options running on browsers and phones, underpinned by a more diverse web of cloud technologies. The architecture of the software has shifted from monolithic and directly-coupled, towards a more interconnected microservice architecture.
However, as HSBC and many other established enterprises have found, the legacy technology hasn’t gone away. In an article in IT Pro Today, Christopher Tossi pointed out that there can be a significant case for retaining at least some legacy technology, on the basis that:
- it already exists
- it already works with the stuff around it
- it has already been vetted in the wild
- it may have been created (often out of past necessity) to run significantly more efficiently than any possible replacement on modern technology
These factors explain why idealism tends to give way to pragmatic reality in enterprises. The notion of a fully modernised technology infrastructure may be desirable, but it is an expensive and complex change, and may simply not be the best business decision in the short or long term to replace every legacy component, particularly if the opportunity cost of that effort reduces the capacity to deliver new, transformative innovations. Mainframe technology, the long-term bedrock of many industries such as banking, is not going away, but is even growing, and remains projected to do so through the rest of this decade and beyond.
What this means in reality is that a seemingly simple transaction such as a personal money transfer may actually result in work being done across an interconnected range of old and new technologies. It is this complexity which creates the problems. But as we’ve seen from HSBC’s observation, and the persuasive arguments of Tossi, that technology is not going away, and it’s not the place we should focus our efforts in solving the “stuck in the middle" problem.
Instead, we need to focus on the two areas which high-maturity organizations have removed from the picture: Organizational Resistance to Change, and a Lack of Automation. The solutions to these issues, the report tells us, will not come from obsessive replacement of legacy technology, but by reducing the fear of change in the company, and by automating around the legacy technology challenge.
