Perhaps the number one problem in enterprise IT is 'change'—how to handle it and how to keep up with a changing world. Gartner says that "IT organizations' application strategies often aren't dynamic enough to handle changes in technology."
We spend a lot of time, effort and money building systems which (if we're lucky) perfectly fit at a single point in time, but then fail or become unsuitable when time moves on and the world doesn't look quite the same as it did in that one perfect deployment instant. we must constantly strive to build systems that are not fragile in this way.
A major contributor to fragility is 'coupling', the way that dependencies get built into our systems at all levels—often this is inadvertent. Coders are familiar with technical coupling such as the way that a software program depends on its environment—operating system, hostname, network address, dynamically linked libraries etc. But it turns out that coupling goes all the way through our systems and the enterprise and often has nothing to do with technology. For example:
- Software applications specified and purchased at a departmental level contain assumptions about the way that department performs it's function now, but may not reflect the needs of other departments or other times.
- Software vendors release and maintain applications which run standalone with a uniform configuration across thousands of customers regardless of those customers individual proclivities and environments.
- One department customizes a shared application resulting in overhead in terms of requirements management, stakeholder management, scheduling downtime, cutovers and migration.
Most enterprises are very application-centric and there is a problem in that applications are difficult to change. Database schemas, object models, user interfaces and programming interfaces are all tightly coupled. Add a new field to a database table and if you're lucky, the change ripples through the whole system. More often the change must be manually copied across the whole system. Because applications are hard to change, baking business rules or processes into applications does not make for business agility.
This is where a layered architecture brings flexibility as to where and how IT systems change. Layering is the ability to separate key enterprise functions into different logical locations where they can be executed, managed and changed with relative independence.
The figure below illustrates how a layered architecture supports change. The "Y Axis" represents the continuum between technology (hardware, network, bits and bytes) at the bottom and business (the value chain) at the top. Roughly correlated with this continuum is a measure of the "pace of change" which is illustrated on the right of the diagram.
Applications are hard to change and therefore have a low pace of change. Conversely the business level changes often and has a high pace of change.
I'm aware that "pace of change" has a double meaning here. First there is the "ability to change." We know from experience that application change is hard and hence applications have a low ability to change. The other meaning relates to the "desire to change." Businesses desire change in their drive to keep up with changing market conditions and consumer behaviour. The fundamental problem arises when a desire for a high pace of change is hindered by a low ability to change.
If a fast changing business is "glued" directly to slow changing applications then cracks inevitably appear. Introducing additional layers into the architecture can fill the gap between applications and the business to help manage change.
The second layer in the diagram externalises application capabilities—functions, events and data through services (or Web APIs) and events. Services and events usually have a strong affinity with a particular application. For example services and events relating to customers are often supported by a customer relationship management system. This is why that layer lies adjacent to the application layer. However, done properly, services and events have a closer alignment with the business than application interfaces and also have a higher pace of change than applications, which is why this layer sits one level above the application layer.
The next layer up holds architectural elements which have a closer affinity with the business: processes, rules and insights through analytics. These elements are supported by the availability of services and events in the lower layer, and in turn directly support the business value chain in the higher layer. Elements at this level have a relatively high desire for change and are best implemented using tools that provide the ability to change, such as business process engines, business rules engines, event processing engines and dashboards.
This concept of layering is similar to Gartner's Pace Layered Application Strategy which separates an application portfolio into different layers depending on their pace of change. "Systems of Record" have the lowest pace of change and represent stable business capabilities. Above this lies a layer of "Systems of Differentiation" which represent activities and capabilities unique to the organization. "Systems of Innovation" are in the top layer where short-lived experimentation happens. Gartner's advise is that application funding, governance and architecture should be different at each of these layers in recognition and support of their different pace of change. The architectural components we put in the middle—services, events, processes etc.—sit squarely among Systems of Differentiation and enable the Systems of Innovation.
I've presented here a highly simplified view of a layered architecture and there are details within each "box" that we'll drill into at a later time. The fundamental idea though is that business value and business agility are supported by a planned and managed IT architecture. Application portfolios have traditionally obtained the lion's share of attention, but it is the things that "fill up the spaces" between applications that provide value and differentiation.