Architectural Patterns

TRELLIS is based on industry best practice, using patterns and templates that create open-ended solutions that are easy to evolve, extend and support.

Current architectural patterns build upon best practice software development approaches that have evolved and matured over the past 20-40 years:

Each of these approaches has aimed to simplify the software development process, by creating smaller components that can be unit tested and then integrated into a more complex solution.

The technology and scope/size of systems has grown significantly over time, but many of the fundamental principles remain unchanged. Each new generation of developers rebrands the methodology, but when you strip away the hype you always get back to a basic pattern of interacting services/activities communicating via synchronous/asynchronous messages and utilizing data pools/storage.

As we move to more distributed systems and hybrid cloud solutions, endpoint security and extensive monitoring become real concerns. Microsoft has addressed these concerns at the platform level with Azure Active Directory and Log Analytics (OMS). Microsoft’s tooling holistically spans the full scope of hybrid solutions from on-premises operating systems to cloud services delivered on IaaS, PaaS or SaaS.

MASCOT - Activity Channel Pool

MASCOT (Modular Approach to Software Construction Operation and Test) was developed in the 1970s to describe complex computer software operated on distributed networks.

The basic communication design was based on asynchronous Message Exchange Patterns that remain largely unchanged after 40 years.

TRELLIS designs use variations of ACP diagrams when describing data flows across organization boundaries.

Data Flow Diagrams (DFD) (Yourdon & Constantine 1979)

Data Flow Diagrams are an elaboration of the basic concepts of software design. They also focus on developing software using a modular process approach, with data flowing between processes and being stored in data pools. The book expanded on MASCOT by allowing for both synchronous and asynchronous data flows, and contained some of the best early work on coupling and cohesion.

TRELLIS designs use variations of DFD diagrams when describing data flows across organization boundaries.

Object Oriented Design (Booch, UML 1990s)

This was a restructuring of previous software development architectures based around the emergence of object-oriented languages like C++ and Ada. As software came to be encapsulated within classes, object-oriented design methodologies evolved which emphasized the separation of interface from implementation. This led to separate versioning of interface and implementation code, and many of the fundamental design considerations for modern services.

TRELLIS designs use variations of UML modeling diagrams when describing solution requirements and design of class libraries.

Service Oriented Architecture (SOA Erl 2004)

TRELLIS makes extensive use of SOA principles and pattern, as the entire TRELLIS framework is deployed as a set of interacting services.

Although there is no industry accepted standard that defines Service Oriented Architecture, many people would accept the high-level approach where services are roughly decomposed into five groups:

There is no industry standard for implementing a Service Oriented Architecture, and GreatIdeaz utilizes multiple techniques and pre-built components to build services determined by unique business need. Implementation techniques for Service Oriented Architectures include (but are not limited to):

Service Contract and Endpoints

All services provide a clearly defined and version-controlled interface called the Service Contract that permits other services and applications invoke the encapsulated functionality.

The service contract exposes multiple endpoints that can invoke functionality encapsulated by the service in either synchronous or asynchronous modes.

To promote stable integrations, the service contract needs to be formally managed and version controlled so that ad-hoc changes do not break existing integrations. A new version of the service contract would be required if an existing endpoint is removed or modified. Adding additional endpoints to a service contract may be done without creating a new version, but this is not recommended.

Each Service Contract and Endpoint should be secured both at a functional level and at the level of the underlying data being transformed. Credentials should be enforced at the point that the endpoint is invoked and then passed into the implementation to control access to underlying functionality and data.

Service Implementation

The implementation of a service should be a “black box” whose internal details are not visible or available to other services. This keeps services loosely coupled, and ensures that integrations are not broken if the underlying implementation of a service is changed.

Black box implementation also decouples the version of the Service Contract from the version of the Service Implementation. This enables the upgrade or replacement of an underlying application without breaking an integration.

The implementation should be isolated such that a failure is contained within the service, and an appropriate error response is returned to the client. Such failures should be isolated to a request and should not impact other requests currently being processed.

Service Boundaries

A service should fully encapsulate the functionality that it exposes.

Service State

Where possible, service contracts should provide stateless endpoints to promote smooth scalability. If maintaining service state is absolutely required, the state should be stored in a shared location (such as a database) that can survive the failure of the implementation container.

Service Composition and Hierarchy

Services can also exist as part of a hierarchy of functionality where higher-level services compose the functionality of lower-level services as part of an integration workflow.

Service Granularity and Normalization

Service Contracts need to be defined at an appropriate level of granularity to minimize the number of endpoints exposed. For example, if the service synchronizes Customer data, there should be one endpoint that contains all customer data and not multiple endpoints for each customer data element.

On the flip side – there should be sufficient endpoints to provide relevant functionality at a granular level to build workflow and promote reusability.

It is usual to see a period of normalization as services are decomposed or consolidated to minimize redundancy and optimize performance.

Service Governance and Discoverability

Service Contracts should be subject to formal version control to promote stable integration, and documented in a directory shared with integration partners to build services that span multiple enterprises for EAI/EDI solutions.

Enterprise Service Bus (ESB Chappel 2004)

The Enterprise Service Bus pattern is supported in different ways by multiple platforms. It is based around a shared message passing system (the service bus). Applications connect to the service bus to exchange messages to create and transform data. Some service bus platforms provide orchestration engines that can trigger, control and monitor workflow when messages are transmitted.

Application connectors abstract the details of the application implementation and version to promote stable integration of functionality. Each of the connected applications provides a System of Record (SoR) for some of the enterprise data. Modifications to data in other applications generate update messages to the SoR application which is then responsible for updating the data.

Service bus platforms can be used across enterprises to support EAI/EDI. Each enterprise may use its own platform for a service bus with gateways between different enterprises handling inter-site communication and translation.

TRELLIS uses the Azure Service Bus to provide the core messaging system. The Azure Service Bus supports both Message Queue (many to one) and Topics (Publish/Subscribe many to many) message flow patterns.

Application connectors are typically implemented with Azure API Apps. GreatIdeaz recommends a restricted use of the REST pattern where only POST requests over HTTPS are allowed when accessing enterprise data. This prevents the exposure of enterprise key/ID data in URL strings.

Orchestration is typically implemented with Azure Logic Apps or BizTalk servers.

Message formats are usually defined (and validated/enforced) using XSD. Message data is then transmitted in XML. Within the service bus, translation of messages between connectors is handled with XSLT.

Micro-services Architecture and Domain Driven Design (DDD)

This architectural pattern is an implementation refinement of Service Oriented Architecture. It is a relatively new architectural style (2011) where applications are structured from a collection of loosely coupled fine grained services with light weight protocols. Services are constantly refactored and deployed through continuous delivery.

Micro-service applications are composed of small, independently versioned, and scalable customer-focused services that communicate with each other over standard protocols with well-defined interfaces.