before you read... this is a collection of thoughts. A ramble that I have pieced together from notes from my black book... Nevertheless, I hope there is some value.
Building a genuinely adaptable platform requires foresight and planning. If you are an incumbent, the opportunity to redesign the architecture to match the new-world-tech and design principles is a mammoth undertaking. If one is so bold enough to take up the challenge, the options are to refactor & re-engineer or build from scratch & discard. In our experience, the overall effort seldom differs. Hence we fluctuate towards the later as it gives you more of an opportunity to transform the platform/business. We are not talking about 40-year-old+ retail banks; We see this in 12-year-old micro-lenders and some startups who have just started to trade.
Ever wondered why?
These platforms are built by incredibly talented people with the right intentions. The chance of an engineer designing a solution that fixes one issue but creates five is rare (even though, I have seen this happen). The problem almost always lies in the environment.
In house software, teams are always under pressure to deliver against a set of business KPIs. The priority is set based on the near-term revenue, growth and targets. The long-term health of the platform and the customer experience -not just reskinning, real features that add tangible value to the customer- will inevitably be deprioritised.
Above is the critical challenge that all in-house software teams face, especially if your team quality and size is below ideal. It is hard to argue against building proprietary solutions that extend business IP. It is easier to manage, one feels more in control, no dependency on third parties and the investors love it. However, you will end up building 95% of the platform to deliver the 5% that differentiate you from the competition.
Decoupling a platform from business KPIs seldom 'fly'. To achieve the constant value-add against the competition, the company will drive the engineering teams to deliver quicker. At this point, the aforementioned foresight and planning required to build adaptable software solutions will be seen as a 'blocker'. The architectural design principles that everyone embraced at the start of the year will be thrown out. "Let's just deliver this... Let's worry about the tech debt later.." I have heard many utter.
As a result, the adaptability of the platform across all layers will steadily degrade, making it harder to introduce new features. And more importantly, the intangible accumulation of technical debt will slowly grind the business to a halt.
I remember being part of a product team at a leading bank who has designed a mortgage solution that did not connect to the main customer table (Think about it!!). As a result, for each new mortgage application, all customer data has to be manually entered by the staff and stored in separate data structures via a new system. Whist, it might have seemed to be a good idea at the time to deprioritise such functionality to deliver on time, the downstream impact was undesirable. The complexity that this introduced to customer portals, online banking, mobile app banking, CLTV reporting was immense. As a result, to create a consistent customer view across the mobile app and the online banking platform, a new matching engine was introduced which incrementally matched 6 data points to ascertain if the customer is the same person or not. Introduction of a simple foreign key with the customer identification number would have saved a month of engineering and two months of arguing about the remedial solution.
Building adaptable platforms are not tricky. It requires foresight, planning, upfront investment and discipline. Such a platform follows a component-based architecture that has clear mounting points. I/O messages connect to each mounting point through pipes and are never bypassed to save time or money. This technique allows one to extend the functionality of the components over time. Even replace the parts altogether to a new tech stack without having to face prolonged shutdowns or 'big bang' migrations. Most importantly, by adhering to a simple set of protocols, the latest technology can work with the old.
While it is impossible to create a solution that does not incur technical debt over time, a component-based architecture can vastly reduce the inefficiencies and revenue loss resulted from a platform marred in tech-debt. A component-based design pattern will allow a platform to be evolved by introducing incremental change and deploying (sub)components independently.
Thanks to Dikaios Papadogkonas (https://www.linkedin.com/in/dikaios-papadogkonas-b007614/) for helping with brainstorming ideas.