This post will discuss the various types of activity information that we may want to record from applications executing in an SOA environment. At this stage we are not concerned with how these events will be described in the activity event schema. We will also want to allow for extensibility, to enable users to define their own additional event types - so this thread is simply aimed at defining the initial scope of events that we should consider supporting.
The first set I want to deal with are the ones related to interactions between distributed components in the SOA:
1) Service Invocation
The invocation of a service, and any subsequent response can be viewed in terms of four separate events:
sending the request
receiving the request
returning a response or fault
receiving a response or fault
As these events are representing the RPC pattern, there would only be expected a single receive event for each send. These events will be associated with a service interface and operation, and therefore are distinguished from the following more general message passing events ....
2) Message Exchange
These can be described using two activity event types:
In this situation, there is no RPC style constraint, and therefore it will be possible to support other message oriented patterns, such as broadcast/multicast - so each send may result in zero or more receives.
Although service invocations can be viewed more generically as simply message exchanges, when analysing activity patterns, it would be useful to have some additional understanding of the interaction semantics. Its possible this could be encoded in the activity event information, but given that we are primarily targetting an SOA environment, then this distinction seems appropriate?
The main reason for initially focusing on interaction based events, is that they will be used to tie together the activities that occur within each of the services, to provide an overall view of the activity associated with a business transaction.
Other activity events, such as those that might be generated by components in an ESB, or processes within a BPM, will be discussed in subsequent entries on this thread. When considering what are suitable events, we need to have a view on how the information will be used. Generally we need to be looking for information that can help an organisation understand how often business relevant tasks are performed, and how long they take to perform, as this can be used to tune how the overall business transaction is executed.
As an addendum to the discussion on service interaction based events, we need to consider how this will work within an ESB, such as Switchyard, which is based on SCA.
Potentially all internal communications between the SCA components could be viewed as service invocations, so do we want to record each of these as activity events? Or are we only interested in invocations that only occur between the switchyard app and other external parties (i.e. go through a binding)?
My initial thoughts are that it shouldn't really matter whether these services are internal or external. They represent the invocation on a service interface, and the co-location of services is just really a packaging issue. There will be cases where the communications between 'internal' services are just an implementation detail, and those internal services are not really important, but in these cases it may be possible to filter out those service events at some level.
The next set of events for consideration is related to BPM. As an example, we can look at the events generated by the Apache ODE BPEL engine (http://ode.apache.org/ode-execution-events.html) which is used within the Riftsaw project, and therefore will be part of the Switchyard ESB.
As indicated by the types of event generated by ODE, with BPM there are a few areas of interest:
- Process instance lifecycle - at the outer level, we want to know when a new process instance has been created, and when it completes (whether successfully or unsuccessfully). The process instance id, along with the process definition name, can be used to scan for other events (potentially across other business transactions) that related to this process instance.
- Activity lifecycle - as it will not be possible to know all the types of activity a BPM system will use, having a general understanding of an activity's lifecycle (start, end) with a reference to the activity within the process definition, means that tooling can perform analysis using a combination of the events and the process definition. The start/end markers provide delimiters to understand what other activities were performed within the scope of the task.
- Scope lifecycle - may not be relevant for all types of BPM, but in the case of BPEL it is possible to define inner scopes with their own variable definitions, therefore having the ability to record when these scopes start and end may help with the management of state associated with the process instance
- Expression evaluation - can be useful to know the basis for decisions being made in the process
- Variable manipulation - reading and writing of variables, can be used to keep track of state regarding a process instance
- Correlation - ODE provides information about correlation, which could maybe be used to enhance general support that will be required for correlation in the activity monitoring infrastructure
So for BPM, I think the important information that needs to be represented is an idea of scope (i.e. which BPM task is other activities being performed in), delimiters to be able to derive metrics regarding the tasks, references to the process definition to support analysis against that description and state management, to facilitate a form of remote debugging/diagnostics.
Database related activity events:
as with other scoped events, we need to be able to determine how long it took to perform these tasks, either having a 'start'/'end' event for each, or somehow having the duration recorded with the event.
Possibly the same types of activity should also relate to JCA resources?
We should also allow for 'custom' activities - which could be used by the service implementation to record milestones, or durations of interest. Not sure what other activities may be relevant to record from within the implementation of a service? Unless general decision results, etc