Platform Engineering

Saul Caganoff

Crisis of the Monolith

Posted by Saul Caganoff on 15 October 2014

Architecture, Microservices

Last week I gave a presentation at the first meetup for Melbourne Microservices. Since many of us are aware of the general characteristics of microservices I wanted to survey the broader context of forces driving the emergence of microservices as a potential new application architecture.

The IT world is in the midst of shift to the cloud which I think represents a “paradigm shift” that fundamentally changes the way we build and consume IT capabilities. I'm certainly not the only one to notice this, Google's Eric Schmidt talks about the radical changes in business and IT in his new book “How Google Works” and IDC refers to massive growth and disruption fueled by the “third platform“—successor to the current “second platform” paradigm.

“Paradigm” is not just a “wanky” marketing term but an ancient greek word meaning a set of “concepts, values and practices” that represent our view of reality. I've written before about how paradigms relate to the way we work in IT.

Paradigm shifts take a while to iron themselves out. The future of application architecture is not clear yet…it may be microservices or something else. But paradigm shifts do have a repeatable pattern:

  • A crisis phase when problems with the current paradigm build to the point where they can't be ignored.
  • A phase of conflict where different alternatives vie for adoption and struggle against proponents of the old paradigm. Paradigms are resistent to change and failures take a long time to build momentum.
  • The shift where a new candidate emerges and practitioners climb on board. But often the old paradigm doesn't die out completely.

The current crisis is a crisis of “business agility.” The way we build and consume applications simply cannot keep up with the rate of change demanded by business. Change takes too long and costs too much. Some of this paralysis is due to organizational culture and some of it is technical in nature, characterized at the application level as the “big ball of mud.”

There are other problems with the current application paradigm. Scalability is challenging in a world where applications serve a global user-base. How do applications scale to such heavy demand, how do they achieve differential scaling and how do they scale elastically? There is also the matter of commodity versus differentiation where businesses compete using a mixture of commodity off-the-shelf capabilities alongside their own in-house capabilities of differentiation.

In the application integration domain, we've long been aware that the monolithic application seems to operate at the wrong level of abstraction. Individual business capabilities delivered via “services” operate at a better level of both abstraction and granularity—supporting change through remix and re-use. But there is a mismatch between service oriented architecture and the way that applications are currently packaged and managed. Perhaps microservices can address that mismatch.

Complementing these problems are all the opportunities that point the way to the next paradigm:

  • Cloud gives us abundance to support scale and agility at low cost.
  • XaaS (anything as a service) allows us to buy services, not software, to compose new businesses and capabilities.
  • NoSQL databases are cheap and malleable allowing the independent fine-grained backends favoured by the microservices architecture.
  • Continuous Delivery means that change can be automated.
  • Agile has taught us the value of small productive, multi-disciplinary teams.
  • DevOps removes artificial barriers between conception and production.

These problems and opportunities are all the ingredients behind microservices which anticipate a new way to build and consume IT capabilities. But microservices are certainly not a free lunch. In my next post I'll turn to some of the challenges behind microservices which will set the agenda for future discussions.