Microservices and Post-Tonal Architecture

As ever, the technological future would evolve and new revolutions would emerge in ways that defy our anthropocentric expectations. The landscape of artificial intelligence is rapidly being populated by a swarm of highly specialized autonomous agents and not being taken over by these quasi-conscious all-purpose behemoths of cognitive & creative superpowers who would redefine what it means to be human, and who had inspired in us great imagination of dread and anticipation in equal measures. Scientific reality, as they say, is stranger than science fiction.

The primary impetus for these purpose-built smart agents is obviously the efficiency. More well-defined, self-contained and free from external perturbation is the surface of a problem domain,  faster a heuristics-driven technique like statistical inference will comprehensively construct the semiotic rules of that domain or more efficiently a meta-heuristics methodology like Genetic Programming will solve the symbolic regression problem to find a function that describes the data-set of that domain.

Unsurprisingly a similar drive towards fine-grained autonomy is enjoying a high degree of adoption in the conceptualization and implementation of architectural building blocks of large-scale Software (from IT Automation technologies like Application Release Automation or Cloud Management to Social Networking to online Entertainment): Microservices.

The notion of Microservices is a considerable departure from the traditional monumentalist ambitions of Software architecture, which has typically been characterized by a rigorously well-defined blueprint of the entire system to the precise specifications of which the code is expected to be written. If traditional architecture were a centralized compositional technique the Microservices construct is driven by a desire for decentralization, its efficacy lies not in its predictive/anticipatory goals but its malleability. 

In other words, Microservices architecture indicates a movement away from the concept of essential authority or control, away from the notion of a central core of signification and truth. Such a movement away from an immutable set of transcendental truths to a fluid network of interdependence characterizes an epistemological leap of maturity in any field of knowledge. 

For instance, Derrida's playful neologism différance by demonstrating that the meaning of a text arises out of the ever evolving network of relationships -- often in opposition to one another -- embedded in the language and hence such meaning is never static and is always deferred/postponed brings about such an epistemological leap from the notion of transcendental truths that a text was always supposed to carry. 

Such a movement is also championed by the revolutionary insights of Quantum Mechanics -- into the theoretical limits of our observability of the microscopic world and the ambiguities of knowledge about the state of a phenomenon outside observation -- the applications of which have been at the heart of the phenomenal progress in Science and Technology in the last century or so. One of the pioneers of twentieth century Physics, Werner Heisenberg, had summarized wonderfully:"The world thus appears as a complicated tissue of events, in which connections of different kinds alternate or overlap or combine, and thereby determine the texture of the whole." 

The revolutionary leap in compositional theory and techniques (within the tradition of European classical music) from functional tonality up until the late romanticism of Brahms or Mahler to the Serialism pioneered by Schoenberg, Webern, Berg and others offers a highly instructive analogy.  The twelve-tone Serialism of this "second Viennese school" broke away with the supremacy of tonal centers and thereby opening up the combinatorial possibilities of any and all intervals ushered in a new era of musical creativity. Serialism (or later post-Serialism, or simply post-tonal music) continued to evolve beyond the Schoenbergian dodecaphonic music built on tone-rows to ordered sets of intervals, tempo, even dynamics. No longer the absolutes of triads and tonic-dominant and fixed time-signatures but a network of relationships emerging out of personal aesthetics or mathematical formula, where it is the relativity that gives rise to the wonderful sounds (from stark minimalism to punktuelle Musik or "musical pointillism" ) of Stockhausen, or Boulez, or Xenakis. 

We can then borrow the term post-tonal to characterize Microservices based architecture. It is no longer a precisely planned directive of point-to-point communication that drives the flow of control and information, but a choreography of interactions through events that yield condition-driven, use case-optimized pathways of participation between the relevant Microservices. 

An architecture is always evaluated by the value it provides: value to the consumers of the hosted Application or the release product (ease of use, performance, security, auto-configuration, self-upgrade), as well as value to the Engineers building the Software (ease of development, continuous delivery, technological flexibility, heterogeneity of tools and technologies). The efficiency and optimization (to Engineers as well as users) facilitated by a Microservices based architecture depends on the clarity of bounded contexts associated with each Microservice, the topological distribution -- on the functional plane to ensure optimal interactions and in the network plane to ensure minimized latency -- of those Microservices around a hub through which Events are streamed, and the potential of mutation embedded within the design of each Microservice to allow for the behavioral modification in those two aspects in response to changes in external conditions (event stream rate, newer Microservices joining the Ecosystem, change in underlying network topology etc.).

Let us consider a typical Microservice ecosystem of n+1 Microservices distributed around an Event Streaming hub (say a messaging bus like Kafka or ActiveMQ), where some of the Microservices are directly communicating with each other over REST. 

Where the bounded context of each Microservice Mi is defined by a set of APIs (REST/Message envelope) Si.  Let us also annotate any Microservice(Mi)-to-Microservice(Mj) direct communication as Mij, and the set of publish-subscribe messages between a Microservice(Mi) and the Event streaming pipe as Mi(E). 

The fundamental role of a Microservices based architecture would be then to define a series of principles that govern the relationships between these entities. An idealized example of such principles would be

1.  Si ∩ Sj  = Ø or Si; If not null-set then Si = Sj, and they are simply two scaled-up instances of the same underlying bounded context. 
2.  Mij ∈ (Mi(E) ∪ Mj(E)); That is, the point-to-point communications between any two Microservices are only used for the sake of expediency and performance but the underlying service abilities are still made available through Event streaming  APIs. This ensures that a topological reorientation of the Microservices ecosystem would not lead to any loss in functionality. 
3.  Mij ∈ (Si ∪ Sj); That is, all direct interactions between two Microservices are still fully within the scope of their collective bounded context and have no dependency on hidden services that are not formally expressed by the APIs of those services. 

 While such governing principles are essential and invaluable, their efficacy remains a function of how appropriate the demarcation of bounded contexts are (that is, the composition of each set Si) and how versatile and optimized the communication pathways are between the Microservices and Microservices and external agents and services -- all Mij instances and the set of all Mi(E)) -- under different use cases and environmental conditions.  

The combinatorial techniques of composition in post-tonal music offer an inspiration to the art of deciding upon the initial sets of all Si, Mij and Mi(E). The concepts of Prime, Inversion, Retrograde and Retrograde Inversions provide mechanisms to build a more comprehensive ecosystem of solutions starting from a few well-defined autonomous units of Microservice contexts and their interactions.

Can such an ecosystem be seeded with the adequate degree of mutability to allow for evolution from the original sets of bounded contexts and their interaction edges? Does Microservice based architecture facilitate or hinder emergence of self-evolving solutions? To be continued ...