Case Study: Driving Savings With  Platforms and Architecture-Centric Approach to Systems Development

 

 

 

 

 

 

 

 

 “By far the most common mistake is to treat a generic situation as if it were a series of unique events, that is, to be pragmatic when one lacks the generic understanding and principle.”

                                                                                                Peter Ferdinand Drucker, “The Effective Executive”

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Executive summary

 

In the current economic climate, most business executives are struggling with a common problem: how to produce more with less.

Senior IT executives are struggling with their own version of the same question:  how to make a greater portion of their IT budgets available for strategic business innovation and transformation.  Investing in sustainable architecture-centric platform-based approaches should allow companies to spend higher portions of their IT budgets on growing and transforming businesses rather than just on keeping businesses running.   However, building sustainable architecture-centric platforms often requires greater capital outlays up front, execution discipline, and commitment to long-term goals and vision.

 

This case study describes the operations division (the “Division”) of a major financial services firm as it was going through several successive mergers and restructurings. The Division’s leadership team was faced with a need to automate core business functions that previously had been done manually.  These functions included operationalization of risk management rules, including credit and income verification, as well as other due diligence functions[1] essential to this financial institution’s operations.  To automate these critical functions, the leadership team had two viable options: the first, while believed to be a faster way to achieve the end-goal was not based on the software industry best practices for creating architecture-centric application development platforms.  The second option, while requiring additional initial investment, was based on the existing Division’s Application Development Platform (ADP) built according to industry best practices and capable of addressing longer term business and IT concerns in addition to the immediate business problem.   The approach that did not rely on best system development practices was chosen.  The ability of the new system to accommodate some basic changes in the business process as well as capitalize on the investment already made in the Divisional IT portfolio was not given sufficient consideration as it seemed at that point a distant second priority to expedience.  Unfortunately, in the end the entire organization paid a higher price, both economically and in human capital due to this decision.

 

 

 

 

Introduction

 

“Stop Gap” IT solutions, while oftentimes meeting immediate business needs, often result in long term system misalignments and other problems. These system problems are frequently structural in nature and thus can be very costly to fix. The quick fix solutions commonly ignore a need for more resilient[2]  business systems that are enabled by proper application of software engineering best practices.  Unfortunately, the system problems introduced by the quick fix solutions are rarely traced back to the decisions that caused them, and in the absence of robust IT portfolio management that tracks long term results, the same mistakes are repeated again and again.   Resulting IT portfolios often consist of many architecturally incongruent pieces and consequently, each succeeding project is harder to accomplish as maintenance and integration costs continue to grow[3]. 

 

Unfortunately, currently there are no hard and fast rules that can guide us to the best choice in system development. Tradeoffs are numerous, often contradictory and hard to navigate.  The “right choice” depends on many factors, is quite dynamic and is never easy or obvious.  For some situations the “stop gap” is the right choice and for others the architecture-centric platform-based approach is a better pick.  This case study describes the latter situation.

 

Overview of the situation and existing environment

 

A relatively mature[4] IT organization was supporting a division (Division) in a rapidly growing financial services company (Company).

 

Over the previous three years, the Division had invested in development activities that produced the Division’s Application Development Platform (ADP). Major parts of the ADP were its architectural framework, metadata repository as well as robust set of processes including configuration management, and release management.

 

A need for an ORM system has been present for quite some time but recently became critical because the Company had significantly expanded its product/program offering[5], with a corresponding increase in variety and complexity of operational[6] rules. The existing manual processes were unable to keep up with the increased demand for operational business rules management. Earlier, a loan underwriter could review a file manually to assess the risk. But when the number of different loan programs grew, the underwriting team was not able to keep up with the different programs’ variations of rules. For instance, the rule “if secondary financing exists, the secondary financing in the system must match the Hud-1 form” was required for some products/programs but not for the others. Rules like this were hard to remember since they changed frequently and varied by products/programs. Complexity in rules management (and execution) was leading to slower turnarounds and higher error levels.  The Division Business leaders funded development of an Operational Rules Management (ORM) system with the goal of improving the Division’s operational efficiency.

 

Due to a number of factors, a decision was made by the Division leadership to allow development of the ORM system outside of the Division’s Application Development Platform.  The main driver was the assumption that time to value for the system would be shorter and the system itself would be cheaper if the development could be done outside of the Application Development Platform[7].  

 

Unfortunately, in addition to creating another redundant and non-integrated source of system development life cycle (SDLC) information as well as business metadata in the Division, non- ADP-based SDLC processes were used for the ORM system development. This was a completely avoidable problem that has aggravated overall predicament.  Using these ad-hoc processes yielded a stand-alone system that was expensive to align and integrate with the rest of the Division’s IT assets. While some attempts to use existing metadata information were made, the lack of a consistent development platform-based approach, as well as the absence of a minimum set of required framework artifacts, resulted in development of proprietary rules taxonomy and related data models.  Since this proprietary taxonomy was developed in isolation from the ADP, it not only lacked some of the information necessary for its future integration with the ADP, but even the information that was present did not lend itself well for the automated systems integration with the ADP.  

 

The end result was an ORM system that was implemented on technology not supported by the ADP[8] and initially contained roughly 600 rules. The implementation effort took approximately seven months and was successfully deployed to the business unit users.

 

A New Twist -- Merger

 

Shortly after the system was deployed, the Company went through a merger. As a result of this merger, another 500 operational rules needed to be added and the user base had to be increased three-fold.  At that point, it became obvious that the system, if scaled up accordingly, would become increasingly unreliable. A decision was made to port the system to a more scalable technology.  The analysis of the additional rules and development effort took three months, during which time the existing production system was unable to meet the scalability demands of the increased number of users, especially at peak times.  The business losses in productivity due to ORM’s inability to meet deadlines during this period were hard to estimate precisely. However, the approximated cost was comparable to all the additional cost of the development required for ADP compatibility[9].

 

When the new more scalable version was finally deployed three months later on a new platform, it contained only 300 of the required new 500 rules; the analysis of new rules was moving forward extremely slowly.  In the absence of common domain information and process models, it was very difficult to correlate the new rules to the existing base.

The process was also extremely laborious and error prone, and took another two months to complete.  Simple work breakdown analysis suggests that if the ADP-required models existed at the time the merger took place, the development cycle could have likely been cut in half due to the significantly reduced analysis phase. In addition to the extra analysis and development cost, two separated views of the data and thus operational rules were required to coexist within the Division for longer time, creating major inconveniences for the staff often resulted in significant rework and thus low morale.

 

 

 

Sarbanes-Oxley Effect

 

By the time the new system version was under development, the Sarbanes-Oxley (SOX) compliance team became an increasingly important stake holder in the operational rules management process and thus in the ORM system. This development led to a situation wherein every time the SOX or Operations teams needed to update the SOX controls and/or the rules respectively, a manual process had to be executed to reconcile these changes.  The cumulative cost of these manual steps over time also became comparable to the cost of all additional modeling and development had the system been built using architectural framework.

 

Had the ORM system been integrated with the ADP’s metadata repository, the validation against the SOX requirements would have been just a matter of writing a handful of metadata management rules against the existing repository information. Minimal additional cost would have been incurred for incorporating the SOX vis-à-vis operational rules reconciliation requirements since the SOX attribute flag has been part of the architectural metadata repository design from the very beginning.

 

Organizational Restructuring Effect

 

Following the merger, major restructuring led to the split of the centralized business operations team (originally responsible for support of multiple channels) into three separate teams, each dedicated to its own sales channel. In the absence of a robust metadata management repository, it became increasingly difficult to maintain one common set of operational rules for all channels. This in turn led to more redundant activities between the channels. Further, the degree of management activities’ automation started to drop and operations became increasingly manual; additional staff was hired to keep up with the demand.   Although financial information is unavailable, it is fairly obvious that due to the new organizational structure, the cost of manual reconciliation of the SOX controls to the operational processes view (in each of the channels) likely increased at least in direct proportion to the number of operational teams and the number of business staff needed to execute the reconciliation. 

 

 

 

 

 

Conclusion

In the case described above, the additional cost of an architecture-centric solution development (roughly $150,000) would have been recovered in approximately three or four months (please see Figures 1 and 2). Most of the additional cost for the ADP-based architecture-centric solution would have come from work done by IT staff. As one can see from the numbers in “Comparative cost of two development approaches” below, the return on the additional investment turned out to be at least 200% and this is according to a very conservative estimate. 

 

However, ADP would have been used not only by system developers every time the system needed an upgrade and modification thus continue to save valuable resources and shortening time to value, but also for creation of training documentation and by operational staff for actual loan reviews as well as by business knowledge workers[10] in operational support area to answer questions that would cover potentially much broader area than just operational rules execution.[11]

 

It is obvious that there are inherent risks in comparing actual results to a “would be” scenario. However, even admitting that this comparison is retroactive and the author has the benefit of hindsight that enables him to fairly accurately pinpoint savings and other benefits that are advantageous to his viewpoint, it is a reasonable approach since we all know that that unexpected changes are the only certain component of any business this day and age.  Sometimes quick fix solutions are the best way to address an immediate business problem.  However, commonly it is not the only available approach as it was in the case described above since the robust analytical and architectural foundation has been already funded and successfully created.

 

Every project undoubtedly has unique nuances, and this case study presents a view of only one particular set of events.  While expected savings will vary by project, it is very reasonable to expect proportionally similar savings in much larger IT budgets. Choosing platform-based, sustainable, architecture-centric development approaches over more simple and seemingly less expensive solutions may lead to significant cost savings in the longer term if the proper balance between short and long term objectives is achieved. 

 

 

	Activity	Actual	ADP--based Approach	Difference
	Initial system development -- 600 rules and 6 users
	Duration	28 weeks	28 weeks
	Cost	$476,000	$640,000	($150,000)
	Porting to SQL Server and adding 300 rules and 10 users after the merger
	Additional rules analysis
	Duration	13 weeks	6 weeks
	Cost	$130,000	$60,000	$70,000
	Migration to SQL Server, updating existing models and components
	Duration	13 weeks	2 weeks
	Cost	$143,000	$32,000	$111,000
cost	Manual execution of rules management process
	Duration	13 weeks	6 weeks
	Cost	$130,000	$30,000	$100,000
	Additional 200 rules after the merger
	Duration	8 weeks	4 weeks
	Cost	$80,000	$40,000	$50,000
	Manual execution of rules management process
	Duration	8 weeks	None
	Cost	$40,000	$0	$40,000
	SOX Review Process
	Duration	Year-round (2)	None
	Cost	$124,800	$0	$120,000
	Total Cost	$1,123,800	$802,000
			TOTAL SAVINGS	~$322,000

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 1. Comparative cost of two development approaches.

Figure 2. Comparative cost of two development approaches as a graph

 

---

[1] Product, Program & Commitment review rules, Legal review rules, Servicing Rules, etc.

[2] I prefer this term to a more commonly used “flexible”.

[3] For in-depth discussion of the issue please see:  “The Trouble With Enterprise Software”, Cynthia Rettig, MIT/SLOAN Management review, October 1, 2007.  http://sloanreview.mit.edu/the-magazine/articles/2007/fall/49101/the-trouble-with-enterprise-software

[4] CMMI level 3.

[5] The company was initially offering close to 30 different financial products in 6 different risk groups (primarily based on the FICO score of the potential borrower). Those 30 financial products were produced by different combinations of the following characteristics: fixed or adjustable rate, different maturity payment options (5-1, 7-1 balloons, etc), maturity period 10, 15, 20, 30, 40 years, line of credit (LOC), etc.  Additional products came from more rare combinations and options: interest-only loans, payment option adjustable rate mortgages, etc. These new combinations have created approximately 20 additional products.  Each of these products has different operational business rules.  For example, some rules categories are: Appraisal Rules, Credit Rules, Compliance Rules, Legal Rules, Servicing Rules, Product, Program & Commitment, etc.

[6] By operational in this specific case I mean the rules that guide the Division’s operations and established and managed mainly within the Division itself as oppose to rules that may be established at the enterprise  (corporate risk department) or even industry (legal and regulatory compliance) level.  Though separate and distinct these operational rules are frequently derived from the corporate rules and policies.  See footnote 5 above for examples of the rules categories.  

[7] A very important factor in this decision was the presence of the rogue IT team within the business unit itself. This team has no interest in reducing Division’s IT costs and supporting ADP-centric development, since the increase of the ADP role would highly likely reduce the very need in the rogue IT team existence.

[8] The choice was determined mainly by the skillset of the developers working in the rogue IT team managed directly by one of the Division’s business team.

[9] See more financial details in the Figure 1 “Comparative cost of two development approaches” below.

[10] Training business knowledge workers to use robust modeling techniques is an interesting and extremely important topic. However, this topic deserves its own discussion far beyond the limits of this article. 

[11] The rules management is probably just a tip of the iceberg in costs if compared to the benefits of the actual rules execution during the loan reviews and the operational support that is required to maintain such a complex process.