Service components in this zone provide secured access to the Data at Rest across multiple channels which serves as a basis for an omni-channel user experience.
Data presentation components would typically include:
- Native Mobile applications
- Desktop client applications
The role of the Data Presentation components is to simply display the records held in the Data at Rest components.
If the user experience requires business logic to be executed, then the functionality will be delegated to a Data in Action micro-service.
The ideas discussed below will replace our traditional approach to developing user experience and eliminate most of the code associated with the standard Create, Read, Update and Delete (CRUD) processing. For our Insurance Policy example with 100k lines of code, this could eliminate 30-40k lines of user experience code and another 30-40k lines of test code. This should cut our development costs by a further 30-40%. Again, we will improve the quality of the solution we are producing – smaller and simpler solutions are inherently faster and more stable! We can also significantly reduce the time to market for the solution.
Dynamic Page Generation
A consequence of storing business information in a data agnostic Data at Rest service is that we are running our CRUD+ model on XML. This means that we don’t have to tie the user experience to a table/attribute-based data model. The obvious next question is, do we still need to hard code the user experience forms against a fixed data model or can we leverage the XML representation to create user experience on demand?
Traditional Development Approach
When you think about our traditional application development approach, the fact that our user experience overlays a table/attribute data model constrains our implementation choices. Our database CRUD functionality extends beyond the database and into our presentation layer. We will have forms that enable us to search/filter, display and edit each of our concepts. Each of these forms builds on an associated set of business objects to abstract the database. If our data model encompasses 20-30 concepts, we are likely to have 40-60 forms in our user experience – most of the code behind these forms will simply be rendering the underlying data and then writing back updates as the user makes changes. In many applications, the actual business logic can also end up in the form’s code behind intermingled with the presentation. This increases the complexity of the solution and increases dependencies between the data, its representation and usage.
User Experience On Demand
Our Data at Rest service is serving XML so we are no longer bound to a fixed table/attribute data model. We can instead think about user experience in terms of the forms we wish to present. We can create a set of Form Definitions which determine how the XML is represented to the user. Our user experience then becomes a simple matter of matching the XML data with the appropriate Form Definition and rendering a user experience. We can do this by simply associating the ConceptGUID from the instance metadata with the Form Definition used to represent it the two become synonymous.
We can now take maximum advantage of the CRUD+ functionality exposed by our Data at Rest service to provide additional functionality in our user experience. Since our Form Definitions will also be stored in our Data at Rest service they can be exposed to provide the same user experience across, client, web and native mobile applications – all at the same time. This gets around a major versioning issue when release solutions that target multiple platforms and present an omni-channel user experience.
Data Presentation Model
We will use a simple Data Presentation Model to create a user experience that can be dynamically rendered across multiple platforms with minimal code and maximum flexibility. This approach is based on many years background building customised user experience across many project types and platforms. We have abstracted a lot of the common features that demand the creation of custom code and converted them into a data representation of how the user experience should render any XML Instance record from the Data at Rest zone.
Let’s start by addressing the definition of Functionality.
For the Presentation Data Model, we will define Functionality to be the provision of a user experience to a person that will enable them to move data securely between screen and database.
Our Functionality needs to be able to understand the Roles associated with the person and be able to render the user experience accordingly.
Our Functionality needs to render one or more Forms with which the person will interact. TRELLIS has a standardised set of form templates to simplify the creation process based on the type of form. Most forms will be a concept of some type; a simple concept for forms which will describe a single class of objects, or a subform concept for concepts that will cover a set of subclasses. We have colour coded our diagrams to use blue for concepts and green for concepts that need to be sub-classed. We have sub-classed the Form Definition concept itself because we will need to be able to provide forms for different purposes: Applications, Portals, Data Entry, etc.
At its most basic level a Form Definition comprises of a set of Tab Definitions and Field Definitions.
A Tab Definition simply enables us to separate out the fields presented to the person between logical Tabs.
A Field Definition defines what data will be displayed or captured by the Form.
Our Data Presentation service now allows us to begin to render a basic user experience by taking an XML instance record from the Data at Rest service and combining it with a Form Definition XML. For data entry forms (which typically account or 90+% of a business solution) we simply associate the Form Definition GUID with the ConceptGUID of the XML instance record. For other form types, we need a more complex mapping mechanism that we will discuss later.
When we examine our instance record we can map the XML elements to our Field Definitions. This allows us to determine where on the user experience we wish to display or collect data. The type of data, display format, and validation is determined by the Field Definition. Our Field Definition can also be extended to provide online help, tooltips and other descriptive information to help the person correctly collect or understand the data.
We can enrich the basic user experience by considering the considering the security model associated with the instance XML record. We can verify that the person’s user account has been given access to any record and prevent display of any data that the user doesn’t have access to. Here we are enforcing the InstanceAccess table rules established by the Data at Rest service. At the Form Definition level, we also determine who can save changes to the instance record based on user role and instance status.
Our Tab Definitions enable us to selectively hide complete tabs of data from the user experienced based on the instance status and roles assigned to the person’s user account. This provides a coarse-grained approach to providing different functionality for example, on a customer order, the Fulfillment tab may not be visible until the order has been approved.
Within our Field Definition we could also specify fine-grained permissions that specify that certain fields of the record are not displayed to the person based on the state of the instance record or the roles assigned to the user. We can extend this selective Field Permission to limit which fields the person could edit, modify or interact with.
Our Field Definitions are in no way limited to just collecting or reading data, they could also invoke services in the Data in Action zone against data that already resides in the Data at Rest zone.
State Transition Model
Already our Data Presentation Model is providing a complex baseline functionality that can render adaptive user experience that considers the roles and permissions assigned to the person’s user account and the state of the data being presented. Now let’s extend that to include a State Transition Model for the instance data being displayed by the form.
For each Form Definition being used to render a Concept we can identify a set of Allowed States for the Concept. We can extend the definition of the Instances table to include an extra field called InstanceStatus which will represent the current state of an Instance and be restricted to the Allowed States listed by the Form Definition.
Our Form Definition can also maintain a set of Events that can be triggered against any instance of the Concept. We can now combine our Allowed State and Event sets to create a set of State Transitions that can determine how any instance of this concept can move through its lifecycle. For example, an Order may go through the following states: Draft, Waiting for Approval, Fulfillment, Completed.
A typical State Transition will define the Initial and Final States, the Event that triggers the transitions as well as gateway tests to confirm that the Event for the instance and user role. It can also trigger a service in the Data in Action zone either before of after the state change.
In this section we have been colour coding our diagram boxes to identify Concepts that can be sub-classed (green boxes) and those which aren’t (blue boxes).
The idea of being able to sub-class a Concept using a Subform Definition provides a very powerful extension to our user experience.
The essence of a Subform Definition is the ability to add extra Field Definitions to a form (and hence the underlying instance XML) depending on a subtype of the concept.
For example, if the Concept is an Insurance Policy it will have a general user experience with an associated state transition model that will be shared by all Insurance Policies. However, the specific information required for a Life Insurance Policy will require a few extra fields, whilst a Vehicle Insurance Policy will need a few extra different fields.
Using the Form and Subform Definition model allows us to extract the common user experience to the Form Definition and abstract the differences into as many distinct Subform Definitions as we need. It also makes is simple to add more Subform Definitions over time as the needs of the business change.
Documents and Extracts
Once we have collected data from our users we will need to be able to manipulate it to achieve business objectives. Two common requirements are the ability to:
- Produce formatted documents
- Extract data for other systems (e.g. Excel Reports)
We can extend our Data Presentation Model to enable the creation of formatted documents by combining the underlying instance XML with an XSLT file. Again, we can restrict the ability to create documents by the roles assigned to the person’s use account and the current state of the instance record.
We can also add the ability to pull pre-defined record sets from Data at Rest services based on the currently displayed instance record which we can restrict the ability to create documents by the roles assigned to the person’s use account and the current state of the instance record.
Code Generation for CQRS
In the Data at Rest section we discussed optimisation of the database for search and general query. We can now use the Form and Field Definitions to provide baseline data to a code generation process to create the underlying database elements required to optimise the user experience.
At the Form Definition level, we can specify whether we should create read only lookup tables for any specific Concept. If we decide to create a read-only lookup table, we can specify whether to include all elements or a subset of elements based on Field Definitions in the lookup table.
We can then use a code generation service to create any required read-only lookup tables and link them with appropriate population trigger T-SQL to maintain their contents.
Use of code generation based on simple templates and the Form/Field Definitions allows us to create highly optimised query/search functionality without hand writing or testing lots of code. We simply craft and test the templates in detail and then we can re-use them whenever we create or modify functionality. The important point here is NOT to hand modify any code that was generated by code generation. As soon as you start making manual modifications to generated objects you lose the ability to quickly regenerate them as the Data Presentation Model changes.
Localisation and Translation
Increasingly we need to be able to provide our user experiences in multiple languages. In some cases, we may need to provide translations based on the person’s language (e.g. English, French, Spanish) but we may also need to provide “translation” based on type of person who is accessing the application (e.g. medical terms for Doctors and Nurses, common body part terms for Patients). We can include these translations within the Form, Tab, Field Definitions and pull out the required textual form as the user experience is rendered.
Not all platforms have the same capability when it comes to creating a user experience. One dimension of our Form and Field Definitions will provide a mechanism for selecting an appropriate representation for each possible platform. In some cases, it may be a reduced capability, in others it may be to suppress the representation completely.
User Role and Domain Driven Design (DDD)
Consider again our Insurance Policy example: we might want to provide a micro-service that enables “Policy Rating”. Our standard DDD approach is to define a Policy concept strictly in terms of the requirements for Policy Rating. Our business may need to use Policy data in other micro-services such as: Claims Management, Renewal, Accounts, Risk Assessment, etc. In each of these cases the definition of Policy would be slightly different, and each micro-service would store its own representation of Policy. Each micro-service would have to implement its own security model to protect its internal data and synchronise its Policy store with the other micro-services. There are several approaches to data synchronisation, but they can negatively impact performance, increase solution complexity and fragility.
A simpler solution is to adopt the DaaS approach where the Policy records would be converted to XML and stored in the Data at Rest service instead of the Policy Rating service. Each micro-service could add the XML fields it needs to the Policy record, they could also share the “common” fields that would be required by most (if not all) services – for example: Policy Number, Effective Start and End Dates, Policy Type, etc.
The advantages of storing the Policy data once in a centralised System of Record include:
- There is only one copy of the Policy data with CRUD+ functionality
- The latest version of the Policy data is always available to all micro-services in real-time
- Policy data does not have to be copied and cached across multiple micro-services creating consistency issues and service dependencies
- Policy data can be secured with a single authorisation/authentication model
- We can still use DDD, the micro-service just maps the XML it retrieves from the DaaS into its own representation of a Policy and leaves the fields it doesn’t know/care about unchanged
When constructing the Form Definition, we need to be able to consider the role of the person who will be using it. Let’s consider our Insurance Policy example, if the person is a broker Rating the Policy, then they will need to see a different representation of the Policy than a person in the Claims department processing a claim.
In a DDD solution we might have separate Domains and ultimately services for Rating and Claims. The brokers will be using the Rating service and the claims adjusters will be using the Claims service. The two different groups of people will have two different roles but will be accessing the same set of common Policy records. To provide this functionality, we just assign each user to one or more roles and then set up our Form Definition to provide an appropriate representation of a Policy to each user role.
Versioning and Environments
We always recommend maintaining separate environments for development, testing, quality assurance, staging and production. A key take-away for the Data Presentation Model is how we version it and migrate it across environments.
Since the Data Presentation Model is constructed within a single XML hierarchy we can extract it with a single command and place the entire definition of the Functionality within a single XML file. We can now simply version that XML file in any manner we chose. We could use a version numbering scheme, date-based formatting – whatever works for your business.
Loading a specified version of functionality from an XML file will automatically replace the existing definition within that Data at Rest service. When we load a new version of Functionality into a Data at Rest service we should also re-generate the CQRS objects that support it.
Migrating Functionality across environments then simply becomes extract the required Functionality from Development, version it and add to a repository or component library. Then extract the required Functionality version from the repository and load into the selected environment, re-generate the CQRS objects and run any required testing.
When rendering an instance, we should also be considering its state. In our Insurance Policy example, a Policy that is still being rated shouldn’t have the ability to be claimed against.
This means we need to extend the Form Definition to take into account the instance state. Then when we load the instance XML we can use its state to determine its representation. As the state of the instance changes over time so does its representation.
Having a single Form Definition rendering platform means that we need no longer create many separately custom-built forms for all possible combinations of platform, user role, and instance state.
This means that it is easy to change Data Presentation. We sometimes do this in real-time during a requirement gathering meeting - this enables us to be sure that the users are getting exactly the presentation they need to do their jobs effectively.
Having a single Data Presentation service means that we are also deploying fewer components with a shared overall user experience. This makes it easier to move employees between roles – they are essentially using the same application, in the same way, with the same data – just seeing different representations and functionality based on their new role.
It also makes it easier to onboard new resources. There is less software to learn and most of the user experience changes they will need to make can be done in data within impacting any code.