Transaction Business Logic is an important issue, requiring substantial cost and time that can compromise Business Agility.
Rete engines are a well-accepted technology, and provide many of the features required for Transaction Logic.
This article illustrates that ORM integration is the key enabling technology that makes Transaction Logic engines a better match for transaction processing.
Why is Transaction Business Logic important?
Transaction Business Logic is the logic required in processing database transactions to enforce business policies. It is sometimes characterized as Enforcement logic, since the transaction should be rejected if the rules are not passed. Consider the insertion of a new Purchase Order:
- Constraints: the system must verify that the new Customers' Balance is within the Credit Limit; failures result in transaction rollback.
- Derivations: the system must compute the Balance by rolling up the Order Totals; this is derived by rolling up the Line Item Amounts, computed from the Product's Price times Quantity Ordered. This illustrates the notion of chained derivations, where one derivation is based on the result of another, often across tables.
- Actions: the system might take various actions, such as sending email if the balance is close to the Credit Limit, starting a Business Process for order approval, or auditing.
Such Transaction Business Logic complements other forms of Business Logic, such as:
- Presentation Business Logic to retrieve data for display and manage page transitions
- Process Business Logic for workflows, or
- Decision Business Logic for Decisioning.
Business Critical: Bad Data leads to Bad Decisions
The well-known adage "Garbage in, garbage out" well describes the need for Transaction Logic. Considerable time is invested in OLAP (Online Analytics Processing) and Decisioning, both of which rely on valid incoming data. In our simple example, it is clearly unwise to accept Orders that customers cannot afford, or for which the Products are discontinued.
50% of app - Compromises Business Agility
Transaction Business Logic is not a small problem. Even a small problem as described above can require 500 lines of code, representing several weeks of effort. And it is not trivially easy to design for re-use, so that our insertion logic works for changing Purchase Orders, and for Purchase Orders received from Message Processing.
Rete Engines well-accepted for Decisioning
So, Transaction Logic is a large and important systems element - an ideal candidate for automation. Rete Engines are a well-accepted technology, delivering significant business value as described below.
Separating business logic to make it accessible to Business Users, so they can address many of their needs for Decisioning.
Declarative: automated dependencies / ordering
Rete Business Rules are declarative: you simply state the rule expressions, and the Rete engine manages the dependencies / ordering.
Transaction Business Logic Requirements
To analyze the suitability of Rete engines for Transaction Logic, let's identify the requirements for the enforcement of Transaction Business Logic:
Logic for (ordered, multi-table) Derivations, Constraints, Actions
The example above illustrates the need for the categories of logic noted in the title. Derivations are particularly complicated:
Derivations introduce dependencies where 1 derivation can depend on another. Such dependence requires that logic be ordered properly (obviously, we should compute the Customer balance after the Order total is calculated). Dependency Management is complex, since such interactions are an N2 problem.
The "rollup" of Item Amounts to Purchase Orders and Customers is a very common pattern. Since these derivations span tables, the manner in which they are performed can have profound consequences on performance. For example, reasonable alternatives might be:
- Aggregate Queries: issue SQL Select Sum() queries to compute the data. But this is not so simple when the derivation results nest multiple levels, and when the underlying data is itself derived.
- Bring Data into Memory: we might exploit caching to bring data into memory
- Adjustment Logic: we might dramatically reduce database overhead by adjusting a stored total by the change, thus avoiding the aggregate queries
These are not simple design issues. They incur normalization issues, and the design parameters might not be clear until the system is fully implemented at which point it can be very difficult to change.
Enforcement critical: For all changes
The Business need for reliable enforcement is obvious: all changes must be subjected to our organizations rules. The term "Compliance Certification" underscores that this is both important, and not automatic.
In technical terms, this is not trivial. There are many factors, but the following are commonly encountered issues:
- Re-use over transaction source:
Poor design might bind logic to screen functions, requiring replication for message processing.
Experience shows that it might be easy to build insert and delete logic, but the myriad of changes that might occur can make achieving re-use difficult.
- Dependence Complexities, particularly during Maintenance
Programming errors become more and more common as the dependencies grow, so maintenance becomes slow and error-prone.
Transition Logic: old value reference
Transition Logic is a very frequent requirement, where the logic depends on comparing the prior database values to the current ones. This can be important in a number of dimensions:
Many business rules explicitly refer to old values: "salary changes must lie within a given range", "invoke verification processes if balance changes exceed a threshold".
Efficiently designed algorithms can exploit old values to prune database access. For example, if I alter a Purchase Orders' date, there is no need to issue SQLs to verify the customer's balance.
As noted above, an extremely common requirement is rollups, where logic requires the derivation of a sum or count of related data. We also noted that such logic is extremely performance sensitive, for example when the volumes are large, or when the aggregates have chained inter-dependencies.
Declarative: what (not when/how)
We conclude with our most important requirement. Our key objective must be to achieve substantial automation, where the procedural code can be dramatically simplified and reduced.
Such needs are not met by improved General Purpose Languages - they require declarative approaches that simplify with automated dependency management, and reduce code volume by 90% by replacing procedural logic that is 90% "when / how" algorithms with declarative rules.
This may sound aggressive, but we have seen such advances before. SQL represents a dramatic simplification for database access. Even more relevant are spreadsheets, where cell formulas can express complex logic formerly addressed by custom programming.
Gap Analysis: Rete and Transaction Business Logic
Rete engines are an existing, mature technology. While it fills many of the requirements, it also leaves significant gaps:
| 1. Derivation, Action and Constraint Logic
|| Met - Rete if/else rules can perform all the functions
| 2. Automatic Enforcement
|| Not Met - Rete rules must be explicitly called
| 3. Transition Logic
|| Not Met - Rete has no Persistence Integration
| 4. Aggregates
|| Not Met - Rete
| 5. Declarative
|| Met - Rete rules are declarative|
This diagram illustrates that all three short-comings are due to the same underlying root cause: lack of Persistence integration
. You invoke a Rete engine by supplying:
You define Rete rules in named groups called RuleSets, which you supply when you invoke the Rete engine.
You also supply a list of objects. While these can be persisted by Hibernate, the Rete engine does not know that. It simply returns the (possibly altered) POJOs; it is your applications' responsibility to persist these objects.
The lack of persistence integration can be a strength for Decisioning applications - you may simply want to ask "what if" questions without any possibility of persistence.
But this is a fundamental weakness in the context of enforcing Transaction Business Logic, as described in the sub-sections below.
No Automatic Enforcement - explicit calls are required
Effective enforcement is not elective: it must be applied to all updates, regardless of their source. It should not be an applications choice to invoke the Rete logic, since it might forget, specify the wrong Rule Set, etc.
Persistence engines (such as JPA, Hibernate, TopLink, etc) typically provide access to the old values of the submitted rows. Your business logic can use this, as noted above, for logic and/or for pruning.
Poor Aggregate Processing - reloaded, in all cases
It is a well-known issue that aggregate processing is inefficient in Rete engines. The root cause lies in the very model of Rete rule definition, and the data submittal assumptions.
Rete compute values, but the engine is unaware whether the value is the unconditional meaning of the attribute. As such, it cannot analyze changes to the value (eg, new Line Items for an Order), and determine adjustments to the aggregate. Instead, the Rete engine must materialize the aggregate, every time it is referenced... even if the underlying dependent data is not altered. This algorithm is thus un-tuned, and not prune able.
Introducing the need for Transaction Business Logic
The issues above are addressed with the architecture shown here that integrates business logic into the persistence mechanism:
There are a number of reasonable implementations of such an architecture, such as:
- Business Logic is declarative, with support for ordered, multi-table derivations, constraints, and actions
- Business Logic has access to old and new values
- Updates are automatically subjected to Business Logic
In any case, such an architecture addresses the requirements not met by Rete Engines:
- Database Triggers
- ORM such as JPA/Hibernate
- DAO Objects (but hand-crafted, so unclear basis for automation)
Automatic enforcement is achieved through integration with the Persistence API - there is no way for applications to write transactions without business logic execution.
- Transition Logic - inherent support for old values
The invocation of the business logic provides old and new values to the logic enforcement
- Optimized aggregates - adjustment (per old values)
Old values enable the logic to prune SQLs when dependent data is not changes, and to perform adjustment processing to avoid aggregate queries
Ideal implementations will extend basic Declarative provisions for derivations, constraints and actions with supplied action handlers that automate common patterns such as auditing, as well as complex functions like bill of materials processing, and the allocation of goods (e.g., payments) to recipients (e.g, unpaid Purchase Orders). It will often be advantageous to utilize both Decision and Transaction Logic
, e.g., Transaction Logic may invoke a Decision Table.