How do I build a list of Business Rule Engine Selection Requirements?

One of the questions I get a lot is "how do I build a list of requirements" when selecting a business rules engine or business rules management system. These, it seems to me, fall into various categories:

• Rule Syntax - what basic mechanisms exist for editing rules?
• Business User Editing Environment - how easy is it for business users to edit rules?
• Decision Process Design - how can you design a multi-step decision?
• Performance Tuning and Debugging - what reports and tools exist?
• Reporting - how extensive are the reports available?
• Rule storage and versioning - how good is the repository infrastructure?
• Integration - how easily does the system integrate with the rest of your IT infrastructure?
• Dialog Support - how does the system integrate with user interactions?
• Help and Documentation - what is available?

Let's consider each of those in turn:

Rule Syntax

• Is the out of the box rule language understandable by business experts?
• Does the language handle procedural or technical functions without calling out?
• Can the language call on external routines when desired?
• Is the language “friendly” for technical programmers as well as business people?
• Can programmers express concepts clearly and succinctly in their terms?
• Are there succinct ways to input and maintain large groups of rules?
• Are look-up tables supported in multiple formats?
• Are tables limited to single-value numeric return values?
• Are trees supported for dependent rule chains?
• Is there a clear and succinct way to input and maintain relative weightings for many different factors contributing to a decision threshold?
• Can the system track and report the most significant factors in a decision?

Business User Editing Environment

• Can business users enter a rule of any level of complexity through an web-based environment separate from the development environment?
• Do business users require training in programming syntax to create/modify rules?
• Can the editing environment be easily customized to match the look and feel of other editors that business users are familiar with?
• Can the business users be constrained in their choices to enforce safe editing of the rules and to avoid flooding them with too many choices?
• Do changes through the business user editor get versioned?
• Can access to the editing environment be controlled through authorization systems and different views of the rules given to users based on their role and security level?

Decision Process Design

• Is there a capability for separating the procedural order of steps in the decision process from the business logic involved in each step?
• Is there a visual editor for diagramming the order of the steps in the business process?
• Can the “big picture” be created, reviewed, and understood by business people without technical training?
• Can rules be grouped by task in order to simplify maintenance?
• Can related rules be viewed and edited on the same page for ease of reference and context?

Performance Tuning and Debugging

• Can individual steps in an overall decision process be tuned for maximum performance by the execution engine?
• Can you track performance in test and production systems, formatting results for ease of analysis?
• Can you set traces, breakpoints, and watches on all objects and events within the rule project?
• Can you step line by line through the running of the rules, setting a breakpoint on any of the lines of execution?
• Is there a way to visually see the steps that the rule engine goes through and step forwards and backwards to see internal operations?

Reporting

• Can you generate a report on all the components of your project?
• Do comments appear in reports and can you format them?
• Can you find potential rule conflicts?
• Is there an impact analysis report?

Rule storage and versioning

• Does the product allow options in the choice of a storage mechanism for rules (eg: databases, flat files, LDAP directories)?
• Can rule files be versioned and access-controlled?
• Can entire projects be versioned to do release management?
• Is it possible to roll back to previous versions and run the rules as they were at a specific point in time?
• Can the repository be structured to support a hierarchical view of many rules?
• Is it possible to store information about the policy source behind the rule?
• Can metadata of any type be stored on any part of the project?
• Can the repository store information used for rule promotion?
• Is it possible to search a project for a specific term or phrase?
• Can you search based on when the files were edited, who edited them, or any other metadata associated with the project?
• Can you save your queries for later use by others?

Integration

• Can you directly access external data models?
• Can you write rules without defining and maintaining a data dictionary?
• Does the product include deployment facilities for rapid integration of rules into existing production systems, including generation of configuration and rule invocation files?
• Does the product have predefined integrations with application server software, including regular updates to stay up to date with new versions of the application servers?
• Do you have to shut down or restart the production application in order to update rules?
• Can you pass data to and from the rules service without requiring conversion to one of a limited number of supported data types?

Dialog Support

• Are stateful sessions supported for tracking what has occurred to date in a transaction and continuing with session-appropriate questions and dialog?
• Can you group questions into sets for easy interactive retrieval and storage of multiple data items without having to write separate code for each one?
• Can the IDE work with interactive prompts designed for the production system without requiring separate “test only” code substitutions?
• Can you build interactive rules-driven interfaces?

Help and Documentation

• Is context sensitive help available in the IDE?
• Are documentation, examples, tutorials, and help files available in commonly used formats for all product features? Are there aids for both novice and experienced users?
• Is information on best practices included in the documentation?