individual-entry-BA-Blog

« The evolution of BRMS (part 2) | Main | Will Data Drive Decision Improvement? »

An attempt at demystifying CEP, BPM and BRMS

-- Posted by Carole-Ann

The more I read on Complex Event processing (CEP), the more I believe that people are confused or people are trying to confuse end users.  It reminds me of the Business Process Management (BPM) / Business Rules Management Systems (BRMS) market a few years ago: BPM vendors said that the rules could be modeled in the process flow and BRMS vendors said that processes could be implemented as ruleflows...  It took a few years to realize that each technology had a sweet spot and a real purpose.  They were complementary.

Now looking at the CEP Blog from Tim Bass, comments posted here (look at the comments) and there on the blogosphere, it is obvious that the role of CEP is unclear.  I am not disputing the value of the technology but merely begging the industry to keep things simple and straight-forward.

I will provide a simplistic prospective here.  You may argue that it is too simplistic but the more we discuss it, the stronger I feel about it.

Simple CEP / BPM / BRMS model

In our BPM / BRMS world, we came to the conclusion that we needed two types of technology: BRMS tools are used to make business decisions that are owned by the business users, BPM tools are used to carry out a process, executing on the decisions made by the BRMS tool.  Can you make a decision as part of a flow?  Yes.  Is it good practice to maintain them there?  No.  Can you automate a flow in a rules service?  Yes.  Is it good practice to do so?  No.  I assume we are all in agreement so far.

When CEP comes into the picture, we feel compelled to question this model.  Is CEP the right technology for processes?  Is it the right technology for decisions?  Some go as far as questioning whether CEP should replace Business Intelligence (BI) and/or Business Activity Monitoring (BAM).

At that point in time, being a product manager by trade, I get back to the problem definition.  what are we trying to accomplish?  Complex Event Processing...  This is all about processing events.  When is it valuable?  When you receive a flurry of events that cannot be handled efficiently using other traditional methods.  This is actually what happens most of the time in the real world: lots of things happen all the time.  Having a module that processes these events real-time would have tremendous value.  It would help dictate what kind of processes need to be executed and what types of decisions need to be made. 

Do we have to worry about an event overload during the execution of a process?  Typically no.  A single "document" is being treated by the process.  It may access more data when needed but it would not typically get flooded by other external events.  Do we have to worry about an event overload during the execution of a decision step (business rules)? Typically no.  Business rules apply to a set of business objects.  Additional data may be accessed as needed of course.  I qualified my answers with "typically" because there may be times when many events could happens and influence the processing of a flow or a decision.  Would all those "exceptional paths" belong to the process or business rules.  If limited they may but if we are talking about tons of events we may have to deal with, it is more likely we would want to use a module for processing those events, correlating and filtering them.  Isn't that the forte of CEP?

The CEP / BPM / BRMS world seems fairly simple after all:

  • CEP module receives or intercepts a flurry of events and processes them with the objective of figuring out what those events are relevant for; it triggers the appropriate business processes or decision services
  • BPM module receives the request for a given process to be applied to a higher level entity (an application, a document...); it automates the steps defined in the business process
  • BRMS module is invoked with a given context to apply business rules; it makes a business decision 

Carlos Serrano-Morales wrote a great post on that subject.

Compared to the Human Body

My background is in life sciences, a source of inspiration.  Please indulge me here while I digress in a biological comparison.

I picture CEP as being the sensors.  The skin for example receives tons of inputs (events) on temperature, moisture level, etc.  It is not responsible for making a business decision although it happens that it makes "some level" of decision.  Typically raised hair and goosebumps may result from the sensors detecting cold weather.

BPM would be the nerves carrying the information to the brain and executing on its orders.

As you probably guessed, BRMS would be represented here by the brain, capable of making a higher level decision based on the contextual information correlated by the CEP module (it is cold) but also based on business strategies (use a blanket at night or turn up the heater if that's not enough; get a sweater during the day). 

Assuming that we decide here to use a blanket, the BPM system will be responsible for carrying out this order, get the complete system to get up, walk to the linen closet and grab the most appropriate blanket.

CEP may intervene any time to provide input that may alter the process or the decision making.  If the skin feels cold *and* the eye sees that an ice cube was dropped on the skin, the information would be provided again using the appropriate process / decision service.

I want to emphasize that, although CEP was making some level of decision, it is not built or informed to make the business decision.  There is a clear separation of role.

What makes this so complicated?

I think we are all guilty within the industry.  We are used to use our hammer on any kind of nail or screw in order to feel as ubiquitous as possible.  We hate to see technology used for a specific problem, by fear that it loses appeal.  I actually think that success comes from the acknowledgment that a tool has a purpose, and learning how to use it appropriately.  If we were still trying to convince the world that BRMS is a credible alternative to BPM (and vice versa), we may still be involved in half baked projects (product management recommended book: Angel Customers and Demon Customers).  We would not have been able to grow into a recognized component of the agile architecture.  CEP needs to follow this lead.

I am afraid that we are also getting confused by our own algorithms.  Some Business Rules Engines (BRE) are toying with the usage of Rete to CEP problems.  This should be irrelevant to the role of CEP.  If in the end we use Rete, Parallel Rete or any other algorithm to implement CEP, it should not matter to the role of CEP in the first place.  We must admit that our own Charles Forgy has presented his findings at the latest October Rules Fest. The intention was not to create confusion at the usage level - mea culpa if we did.  We believe that it would be a different Decision Automation module, part of Decision Management of course but not necessarily part of BRMS.

Complication comes from other sources as well.  Because CEP deals with events in time, we sometimes associate it with Business Intelligence (BI), thinking that because it can process events real-time, it should be able to process time-aware query (how many A events happened after a B event).  When combining both aspects (real-time and BI), you could build a Business Activity Monitoring (BAM) system but this does not mean that CEP is BAM either...

Well, as passionate as everyone is on that subject, I am expecting a few heated comments here...  This is healthy as it will help the whole industry as well as the end users to come to a conclusion on what CEP is and what it is good for.  Go ahead!

First time on the EDM blog?
Subscribe to the EDM blog feed or check out some other recent posts:

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451629b69e20105361b7f03970b

Listed below are links to weblogs that reference An attempt at demystifying CEP, BPM and BRMS:

Comments

schauecker

This is a very helpful article and I've been happy to read it. Sometimes a text makes the life easier. ;) just now for me to understand better what the difference between CEP and BPM / BRMS is. thx Carole-Ann.

Tim Bass

Dear Carole-Ann,

You are simply (and incorrectly, in my view) reducing CEP to low level event (sensor) processing, which was never the intent of the technology or the architecture.

I covered this in detail in my 8 part series, What is Complex Event Processing?

http://www.thecepblog.com/what-is-complex-event-processing/

and in particular, to analogy to the human body (in Part 1 of 8):

http://www.thecepblog.com/2007/05/14/what-is-complex-event-processing-part-1/

Hope this helps!

Your sincerely, Tim

FICO

Hi Tim,

I expected some reaction. This is ECA (Event Condition Action) at work ;-)

I am pleased that we actually came to the same human body analogy independently. This validates that we are not so far apart.

Reading the CEP definition posted in your blog, it seems that we mostly differ on one aspect: what you call CEP covers most of what we call Decision Management. I see all sorts of technologies being involved in the Decision Automation world, CEP being the "event processing" part (and just that) and "business rules" being another one (and just that) and "business processes" being yet another one (and just that). All of those plus eventually predictive models (you mention neural nets and Bayesian networks for example in your analogy), optimization, etc. contribute to solving decision making challenges aka Decision Automation.

I personally prefer the term Decision Management to CEP for that ecosystem of technologies. The main reason is that we are not only processing events in there. The focus is purely on managing the decisions we make, whether they are reactive to streams of events or scheduled data processing or yet another trigger in another context. The focus is also on the management of those decision in the sense that business users want to own this logic (business rules or processes), etc. I feel that Decision Management for better or for worse is quite descriptive of the task at hand. It also emphasizes the management part rather than the processing part, and the management part is where the key business value is.

More food in the fire...

Carole-Ann Matignon
VP, Product Management at Fair Isaac

FICO

And now some water to my well...

I just came across the Oracle announcement of its entry in the CEP world: http://www.ebizq.net/news/10671.html?rss

I quote: "The solution is designed to enable organizations to detect, filter, analyze and correlate diverse business events".

Carole-Ann Matignon
VP, Product Management at Fair Isaac

Neil

This reminds me of the story about Henry Kissinger, when Nixon first tapped him to be his National Security Advisor. As he was leaving New York, all of his friends from Columbia (U) threw a party for him. In attendance were some of his Washington friends, too. A journalist, overhearing a very loud conversation going on between some of HK's NY friends asked Kissinger, "Why are arguments in academia so loud, emotional and shrill?" Henry shrugged his shoulders and said, "Because the stakes are so low."

These kinds of definitional arguments are sort of tedious. Why can't there be more than one definition of CEP? Who cares? Marketing people, that's all. The question is, do they work and can they solve a problem that needs to be solved? Why not spend more time articulating that so we can put this stuff to work.

-Neil Raden
twitter: nraden

Post a comment

Comments are moderated, and will not appear on this weblog until the author has approved them.

If you have a TypeKey or TypePad account, please Sign In.

Search Site


  • dmblog.fico.com

Subscribe

  • enter your email

Upcoming Events

  • FICO Tools & Analytics User Forum 2012
    BERLIN: September 11-12, 2012 LONDON: September 18-19, 2012 Gain new insights for improving business performance through advanced analytics and decision management tools.