All things Rule!
Estimated Reading Time: 10 minutes
Difficulty: Novice
This post chronicles the journey of creating an Open Source TypeScript Rule Engine from conception to release. While the code will be open source (MIT License), it is asked that this or other articles are referenced, and attributed to Kamau Washington if a strategy or code block is used outside of meshalpha.io repositories.
NOTE : MeshAlpha.io is a work in progress, updates will be found on this site pre-release
Earlier in my career I was introduced to the world of Rule Engines, and immediately found a subject that was far more intriguing and challenging than standard Full-Stack SDLC. Blaze Advisor, BizTalk BRE, ILOG JRules were areas of study, as well as the works of Charles L. Forgy the designer of the RETE algorithm. It took the better part of a year to understand RETE-OO (object oriented RETE), Conflict Resolution, Agendas, Working Memory, Alpha and Beta nodes, discrimination networks, forward chaining, backward chaining, and more. The following year, I completed my first RETE based Rule Engine in C# which I named “MeshAlpha”.
While the Engine was fast, extendable, well documented, and passed nearly every Rule Engine test I could get my hands on, there were a few main issues that stood out :
- An extremely simple DSL (Domain Specific Language) to represent and author rules outside of code, treating Rules as external configuration vs compiled syntax.
- An extremely simple UI that allows Rules to be authored without knowledge of the underlying engine, or with limited to exposure to Rule Engines.
- An extremely simple code base, following patterns and practices that would allow authors besides myself to contribute to the codebase with minimal ramp-up.
Starting over, DSL first
There are many Rule Engines complete with their own DSL, often requiring a significant amount of onboarding for the language itself let alone the Engine ecosystem. This should be avoided at all costs this time around.
PROBLEM STATEMENT
Build a text-based DSL for Querying Facts (Objects, Classes, Models, Interfaces, Types…) using only JSON and or YAML to empower business users, and developers alike. The DSL should be easily readable, pattern based, and allow for hinting in IDEs as well as by schema.
Start small, with a reusable pattern
This DSL should be comprised of repeatable patterns that when parsed and executed alter their behavior based on their location in a query. It should also be familiar to read and write. A pattern that stuck out was S-Expressions or Symbolic Expressions : a tree structured notation of data.
// this is an example of an S-Expression (pseudo code)
(multiply 3 (add 2 5))
This may look very simple, and appear as if functionality would be lost, however this is an extremely simple, and readable notation to express the following :
“multiply three by the sum of two and five”
This can easily by translated into something familiar like JSON
// this is an example of an S-Expression in JSON
[
"multiply"
3,
[
"add",
2,
5
]
]
This then can be easily translated into executable JavaScript
function multiply(...args) {
return args.reduce((total, value) => total * value, 1);
}
function add(...args) {
return args.reduce((total, value) => total + value, 0);
}
console.log(multiply(3,add(2,5))) // = 21
Immediately, the possibilities can be realized!
After running permutations of functions, selectors, constants, etc. the S-Expression seems to be the most robust and simple method of implementing a DSL for MeshAlpha.io’s ION (the Rule Engine). A simple repeatable method signature provides > 80% of needed capabilities to query fact sets.
// this is a simple signature (pseudo code) that provides powerful functionality
type pseudoOperation = [
string, // operation
BaseTypes | PropertyQuery,
...(BaseTypes | PropertyQuery)[]
]
Given the above, I was armed with enough to create the first pass of the DSL, with VS-Code hinting by means of well defined Types and Type Guards in TypeScript. Below is a working POC of the DSL in ION-Query…
// rule-fact-query.json
{
person : {
type : "Person",
where : [
// NOTE* "and" is optional at root
"and",
// firstName is "John", "Jane", or "Jim"
[ "eq", { $self : "firstName"},...["John","Jane","Jim"]],
/*
* lastName matches (Regex) strings ending with variations of "smith" ,"Smith",
* "smithson", "Smithson"
*/
[ "mt", { $self : "lastName"},"[sS]mith(?:son)$"],
// age is greater or equal to 21
[ "gte", { $self : "age"},21]
]
},
address : {
type : "Address",
where : [
// join on personId between address and person
[ "eq", { $person : "personId"},{ $self : "personId"}],
// state is California
[ "eq", { $self : "state"},"CA"],
[
"or",
// city is in "Los Angeles", "Alameda", or "Culver City"
[ "in", { $self : "city"},["Los Angeles","Alameda","Culver City"]],
// addressLine1 starts with 10 or 40
[ "sw", { $self : "addressLine1"},10,40]
]
]
}
}
To be continued…
This was a very challenging exercise that will play a huge part in the creation of the ION-Query library. While this initial step is to prove the theory that a simple readable DSL could be written in JSON, TypeScript, and JavaScript (YAML will be added later), it opens the door to the next steps of supporting provided and custom functions, as well as prototyping the design for the actions supported when a query produces a result from a Fact Set.
Kamau Washington 