Financial Services Data and Analytics Newsletter | November 22

BRE – how does it work?

Business rules are the rules that govern the practices and behaviour of businesses and are executed on a day-today basis in any organisation. These rules are implemented across multiple sources, systems and applications and can be combined with other applications in a complex manner to produce the desired outputs. As the system grows in complexity, a simple rule change that can be defined in a matter of minutes by subject-matter experts (SMEs) or business analysts can take months of programming by developers to get deployed in production, hampering the performance of the business. Due to this, enterprises are turning away from hard-coding business rules and have realised the need for a business rule management system (BRMS).

A BRMS is a technology system used to capture decision logic as a business rule, which is then automated across applications. Instead of embedding rules as code within multiple applications, in a BRMS, the rules are externalised and managed away from application code. This enables the logic to be leveraged by multiple applications and changed independently from the governing applications, which empowers business personnel or developers to manage decision logic as business rules, and not programming code. Moreover, externalised rules are placed in a business rule repository that gives business specialists the ability to create, modify or activate business rules without IT involvement.

The BRE executes all these pre-defined rules or business rule repositories written within a BRMS.

A BRE comprises of following components:

1. business rule repository – a database for storing the business rules as defined by the business users
2. business rule editor – an intuitive user interface that allows business users to define, design, document and edit business rules
3. reporting component – an intuitive user interface that allows business users to query and report existing rules
4. rule engine execution core – the actual programming code that compiles and executes the query.

An example of a BRE being applied to a shopping experience would include configuring a rule of giving a 10% cashback of the total transaction amount to the customer whose one-time shopping expenditure exceeds INR 10,000. Such rules can be configured or revoked easily by business analysts as per their requirements from the graphic user interface itself. However, it is important to clarify that a BRE is not the same as a workflow engine. A workflow engine automates end-to-end multi-layered processes, while a BRE evaluates expressions and decision criteria. Though it is possible to automate multi-step decision workflows along with a BRE, the two are considerably different. Moreover, business rules are a core component of workflow functionality since workflows involve decision-making processes.

CEP – how does it work?

CEP is one of the techniques that is used to identify and analyse data from streams of events – typically arriving in an asynchronous manner – through different sources. This data must be processed ‘on the fly. The CEP technique can apply queries to multiple data streams simultaneously to detect specific patterns of events, thus triggering appropriate actions in real-time. It is used to predict high-level events which are likely to result from specific sets of low-level events. Operational intelligence collects real-time data, analyses cause-and-effect relationships, and correlates it against historical data to provide insights and trigger appropriate actions.

CEP architecture

In the above diagram, a stream of events enters the system through edge devices. The knowledge base holds the domain knowledge that is necessary for interpreting and solving these problems. In a rule-based system, the knowledge gets represented as a set of rules. These events then enter the decision engine and appropriate actions are taken. Events in the decision engine can be processed in cloud and stream modes, with the former being the default mode.

In the cloud mode, events are treated as an unordered cloud. Decision engines in this mode do not consider timestamps, temporal features or any stages of life cycle management. This mode uses the rule constraints to find the matching tuples to activate and execute rules. As long as the rules are written in Java or the rule file , they remain active and get automatically triggered when the rule constraints match. However, events need to be retracted explicitly when they are not needed.

In the stream mode, the decision engine processes events in a chronological manner. The decision engine here takes into account the event timestamp and draws relevance using temporal constraints. Events in the stream mode have specified sliding window times and lengths. The decision engine identifies and creates a proper internal structure at compile time to use data from that sliding window to evaluate the rule and automatically trigger the action events. Moreover, it automatically removes the events when the expiration constraint is specified explicitly.

Difference between CEP and BRE

BRE – use cases

Automated business rules, often used for common real-life scenarios, can help to remove human inconsistency, speed up business processes and eliminate any confusion within such processes. Business rule designers build custom logic, evaluate conditions and choose the expected outcomes. Rules can be low-density (simple rules like basic statements, and sequential rules) or high-density (slightly complex rules like matrix conditions or truth tables), which may need to be combined and intuitively optimised in order to produce the desired outcomes. Moreover, rules can be integrated with most existing systems and used to initiate workflows, assign tasks to actors in the system, dynamically calculate data on reports, automatically produce documents, etc.

CEP – use cases

When latency requirements are low and the volume of events is large, CEP is preferred over BRE. CEP can be leveraged for real-time complex pattern searches, business rules, analytics and cases where best-action recommendations need to be made and the complexity of the size of the data being handled is constantly increasing.

Stock market – improve trading algorithms with CEP

The stock market is one of the ideal use cases for CEP, which can be used to improve trading algorithms. CEP applications in the stock market consist of pattern-recognition algorithms, historical data analysis and processing, price matching, buying and selling triggers leading to decision making, and many more such events.

CEP enables the use of high-frequency trading, which involves placing trades in sub-milliseconds in considerably high volumes. The trading programs based on mathematical formulas or algorithms are used to evaluate stock market rates and trigger the automatic buying or selling of stocks. These programs must be fast, flexible and independent as they execute many different and competing complex events simultaneously to execute complex strategies. The execution is performed within seconds or milliseconds because the profit margin on these trades is so small that the opportunity must not be missed. The higher is the volume and speed of execution, the higher are the profits.

Algorithmic trading components using CEP to execute high frequency trades

Our PoV: Can BRE and CEP coexist in any organisation?

While CEP and BRE are similar in many aspects, the main difference between them is the statefulness of events that allows CEP to retain and process asynchronous events. In addition, CEP encompasses the rule-building functionality of the BRE. Also, BRE is currently more widely adopted in the industry primarily due to tool maturity, in-built frameworks for auditing, logging, monitoring and error handling, and workflow decision-making capability. These features require custom coding or extensive plug-ins on a CEP platform, specifically because many event processing tool stacks are open source and still gaining momentum. CEPs also are known to have other limitations – like lack of a structured language and being resource-intensive – which mandates their enablement on the cloud for effective scale implementation.

Does artificial intelligence (AI) have a role in CEP?

Considering the growth of CEP and AI as a solution offering in the data industry, it is interesting to see how these two technologies can come together. CEP techniques are often used for pre-processing the incoming stream of events before data is fed to downstream applications that employ AI techniques. Conversely, AI/machine learning (ML) interventions are baked in the CEP pipeline – for example, to allocate weightages on low-level events – to determine the probability of a complex event. Thus, the combination of CEP and AI will enable the creation of intelligent real-time analytical applications.