Our recent posts...

Fault handling in Oracle SOA Suite 11g – Part II

This previous blog explained why it is a good idea to address -and handle- business faults separately from technical errors. It also introduced a mechanism used in real life Oracle SOA Suite 11g projects to deal with technical errors in a generic way without having to add this functionality to all our SCA composites again and again. Now it is time to dive into the technical implementation of that mechanism and some nitty gritty details.

First things first: How do we get a hold of these technical errors and how can we determine what to do with them?

Oracle SOA Suite 11g offers a unified fault handling framework for SCA composites and their references, service adapters and components such as BPEL and Mediator components. The framework provides hooks you can use to configure fault handling and possibly call out to your own fault handling code. The unified framework is an improvement compared to the SOA Suite 10g stack that consisted of less integrated components (ESB, BPEL) that had their own fault handling mechanisms. The framework is heavily based on BPEL PM’s 10g fault handling framework.

In SOA Suite 11g you configure the fault handling framework on the level of SCA composites using two files: fault-policies.xml and fault-bindings.xml. By default these files need to be in the same directory as the composite.xml file.

Note that you can place these files somewhere else and have multiple SCA composites point to the same fault handling configuration. MDS is a nice candidate since it is a repository for shared artefacts such as reusable XSD’s, DVM’s, and so on. To do this you need to set the “oracle.composite.faultPolicyFile” and “oracle.composite.faultBindingFile” properties in the composite.xml files and point them to fault binding and policy files in the central MDS location. Whether you use this feature mostly depends on how unique your fault handling per SCA composite will be. For now, we will continue with the basic scenario in which we define fault policies per SCA composite.

First of all we will configure the fault-bindings.xml file. This file defines what elements are bound to what fault policy. Elements can be components, references, service adapters or an entire composite. The actual fault policy that is referred to will be defined later on in the fault-policies.xml file. Since business faults can be dealt with using BPEL activities such as Throw and Catch activities we want to have all remaining faults (all unexpected faults) in the entire composite to be handled the same way.

Let’s say we have a simple SCA composite with an inbound file adapter called “MyInboundFileService” and some other components such as a BPEL and Mediator components. Our fault-bindings.xml file could look like the following:

<?xml version=”1.0″ encoding=”UTF-8″?>
<faultPolicyBindings version=”2.0.1″ xmlns=”http://schemas.oracle.com/bpel/faultpolicy”>
<composite faultPolicy=”MyCompositeFaultPolicy”/>
</faultPolicyBindings>

In this example we bind fault handling for the entire composite to the -yet to be defined- policy “MyCompositeFaultPolicy”. Instead of the “composite” element you can use the “component” or “reference” elements to apply fault handling on a more granular level.

Next we need to define the fault-policies.xml file. This file defines the actual policies and the conditions when these policies should be executed.

Following the example we will define a single policy, namely “MyCompositeFaultPolicy”:

<?xml version=”1.0″?>
<faultPolicies xmlns=”http://schemas.oracle.com/bpel/faultpolicy”>
<faultPolicy version=”2.0.1″ id=”MyCompositeFaultPolicy”>
<Conditions>
<faultName xmlns:medns=”http://schemas.oracle.com/mediator/faults”
name=”medns:mediatorFault”>
<condition>
<action ref=”ora-custom”/>
</condition>
</faultName>
<faultName xmlns:bpelx=”http://schemas.oracle.com/bpel/extension” name=”bpelx:remoteFault”>
<condition>
<action ref=”ora-custom”/>
</condition>
</faultName>
<faultName xmlns:bpelx=”http://schemas.oracle.com/bpel/extension”
name=”bpelx:bindingFault”>
<condition>
<action ref=”ora-custom”/>
</condition>
</faultName>
<faultName xmlns:bpelx=”http://schemas.oracle.com/bpel/extension” name=”bpelx:runtimeFault”>
<condition>
<action ref=”ora-custom”/>
</condition>
</faultName>
</Conditions>
<Actions>
<Action id=”ora-terminate”>
<abort/>
</Action>
<Action id=”ora-custom”>
<javaAction className=”nl.approach.MyFaultPolicyJavaAction” defaultAction=”ora-terminate”>
<returnValue value=”ora-terminate” ref=”ora-terminate”/>
</javaAction>
</Action>
</Actions>
</faultPolicy>
</faultPolicies>

As you can see from the example we first define the criteria when the policy should be executed. In this case we want it to be executed in case of any technical error. More specifically in case the error is of type “mediatorFault”, “bindingFault” or “runtimeFault”. Note that we can define more intelligent conditions that can be content-based (e.g. based on process instance variables).

When the error meets any of these criteria the actions within the “Actions” element will be executed. Instead of configuring default actions such as abort, retry or rethrow we redirect the fault to our own Java class called “MyFaultPolicyJavaAction”. This is allowed as long as such a class implements the “IFaultRecoveryJavaClass” class containing the methods “handleFault” and “handleRetrySuccess”. Since the fault may occur within synchronous processes the fault handling framework needs to know what to do after it delegates the fault to some external piece of code. In order to do so the “handleFault” method needs to return the outcome as String. This outcome should map to a predefined fault action. In our example we abort the process instance after our custom Java class has been executed by returning “ora-terminate” that is mapped to the default abort action. Next to that, Java actions need to define a “defaultAction” attribute in case the outcome cannot be mapped to a predefined fault policy.

For some reason rejected messages need to be defined separately. In other words, such faults remain uncaught when using the above fault handling configuration. An example of a rejected message can be an inbound file that cannot be parsed correctly by a File Adapter. To have rejected messages handled we need to specifically include it using the exact name of the adapter service or reference. In our case the inbound file adapter is named “MyInboundFileService”. Our fault-bindings.xml file now looks like this:

<?xml version=”1.0″ encoding=”UTF-8″?>
<faultPolicyBindings version=”2.0.1″ xmlns=”http://schemas.oracle.com/bpel/faultpolicy”>
<composite faultPolicy=”MyCompositeFaultPolicy”/>
<service faultPolicy=”RejectedMessages”>
<name>MyInboundFileService</name>
</service>
</faultPolicyBindings>

Note that you can add more than one adapter name to the “service” element. That way all rejected messages of all adapters can be handled the same way. So for instance you can add “MyOutboundDatabaseService” to the “RejectedMessages” policy too.

We need to add a fault policy to the fault-policies.xml file so our Java class is executed:

<faultPolicy version=”2.0.1″ id=”RejectedMessages”>
<Conditions>
<faultName xmlns:rjm=”http://schemas.oracle.com/sca/rejectedmessages” name=”rjm:MyInboundFileService”>
<condition>
<action ref=”ora-custom”/>
</condition>
</faultName>
</Conditions>
<Actions>
<Action id=”ora-terminate”>
<abort/>
</Action>
<Action id=”ora-writeToFile”>
<fileAction>
<location>/my_messages/rejected_messages</location>
<fileName>my_message__%ID%.xml</fileName>
</fileAction>
</Action>
<Action id=”ora-custom”>
<javaAction className=”nl.approach.MyFaultPolicyJavaAction”
defaultAction=”ora-terminate”>
<returnValue value=”ora-terminate” ref=”ora-terminate”/>
</javaAction>
</Action>
</Actions>
</faultPolicy>

In the next blog we will dive into the Java class and see how it handles the fault. You can read more about fault handling on OTN.

Experiencing coaching an user experience graduate

Excitement…

The moment I was asked to coach an User Experience graduate from the Hogeschool Rotterdam, I was enthusiastic. In my professional career I’ve been coaching several students and to me this has always been very inspiring. When you are working in the area of User Experience for quite some years like me, working together with students provides me with new viewpoints and insights. These new insights stimulate me to look differently to my own assignments and that is very refreshing.

Feeling of insufficiency…

This time there was one difference with all the other student projects I’d coached before. This specific graduation assignment was defined by my UX colleague at Approach without my consolidation, because I was involved in other assignments at that time. Thereby this graduation project consisted of a topic of which I had very little knowledge. This definitely was a hurdle to take, especially at the start of the project. Regarding the content I wasn’t able to give the graduate much of advice.  That felt as a shortcoming on my behalf.

Fortunately Approach employs more professionals, who were able to help the graduate with his topic. By asking relevant and provocative questions, we tried to stimulate him to focus his research and come up with refreshing insights and conclusions.

Tip: As a graduation coach you don’t need to know all about the subject. In the end the graduate will be more experienced on the topic than you are anyway. However with your knowledge and experience, you can still stimulate the graduate to explore all far corners of the subject by asking the right questions.

No sense of urgency…

It’s in a sponsor’s interest to keep the graduate on the right track regarding the planning and deliverables within a graduation project which should come to an end within a specific amount of months. Some graduates might find this quite a difficult task to accomplish, possibly because their school assignments mostly have predefined milestones and shorter time spans. Therefore I requested him to set up a planning. Being his sponsor from Approach, I wanted him to think about what he expected to deliver to me. I also asked him when he would deliver. Due to my experience in coaching other graduation projects, I noticed that for graduates, content often is more important than planning. In a business however as well content as planning are both of equal importances. If the agreed content isn’t delivered in time, it might not be useful/relevant/valid anymore.

Tip: As a graduation coach you need to stimulate (help) the graduate to set up a planning including deliverables and liaisons. The planning should be fine-tuned during the graduation process and deadlines should be kept.

Desperateness…

I expected/asked the graduate to keep me informed about his way of working and his progress. Most of the working hours we were sitting in the same office, so I expected communication would not be a problem.

During their education UX students appear to learn a lot about gathering content and applying design methods. However they seem to have hardly any experience in dealing with a sponsor and the related communication expectations.

Despite of, maybe even due to, all my experience with experienced professionals, I didn’t realize that. I waited in vain until the graduate would keep me posted about the outcomes and hands-on deliverables. I repeatedly asked for information, but it didn’t help. Like this, I didn’t get any information. Instead, I should have planned a regular meeting, at least once every two weeks.

Fortunately, social media can be of great help. I started to follow my graduate on Twitter. I tried to figure out what the problem was. I gave tips and tricks to support his way of working. I gave him deadlines. I felt I wasn’t able to coach him properly. I had become a policewoman.

Tip: In case of a graduation project it sometimes is better to assign the coaching and the sponsoring task to two different people within the guiding company. Then the coaching professional can focus on coaching, while the sponsor plays the role of the ordering customer.

Relief…

Together with my UX colleague Nils Vergeer within Approach, we each took on one task. This worked great. Nils told the graduate what he as a sponsor expected him to deliver at what time and told him what would happen if he didn’t. In the meantime I could go on supporting the graduation process and supply him with tips and tricks on how to deliver in time.

Tip: As a sponsor you have to be clear in expressing your professional expectations according to deliverables and planning.

Joy…

This worked remarkably well. The graduate found his way and graduated in time by presenting a great report on the subject, supported by a tool he designed to support UX-designers and software developers working closely together.

Fault handling in Oracle SOA Suite 11g

You generally want to differentiate between technical errors and functional faults within your processes and services. Functional faults are those that have meaning to the business and might be expected. Functional faults and handling these faults can be part of a process. Consider the example of electronic invoice handling in which an invoice is processed that has a total amount of $2000 while an organization only approved an amount of $1500. In this scenario we can use a human task to halt this particular process instance and assign it to the finance department. An employee of the finance department acquires the task and investigates the issue. He or she may conclude that the client sending the invoice was mistaken, that the invoice approval was not entered correctly in our backend IT-systems or that someone put a coffee mug on the invoice and hence the amount was wrongly interpreted by our scanning and OCR software. In any case, after this human intervention the process may continue again and follow the “happy flow” in our BPEL or BPM processes.

When it comes to technical faults you probably do not want to design error handling in the process itself. If you do, your processes and services will end up being cluttered with all kinds of additional process logic such as while loops, gotos, catches, event handling, and so on to try to recover from technical errors. Technical errors might not be recoverable at all; think of an invoice file that is incorrectly formatted, an invoice file that contains negative numbers while your service or process only accepts positive values, or an invoice file that is mangled during transport. Besides, trying to handle these errors makes your SCA composites look like a mix of spaghetti and circuit boards. Not exactly flexible, agile and manageable: the things we wanted to achieve with service- and process-orientation in the first place.

This blog series contains a possible mechanism to generically handle technical errors in your processes and services -that are wrapped as SCA composites- in Oracle SOA Suite 11g.

In one of our projects we came across a scenario in which administrators need to be notified in case of technical errors in any of the SCA composites. Next to the notification they want the corresponding composite to be terminated. Administrators then investigate the cause of the problem and possibly restart the process instances that are involved. Since every employee uses a task-driven portal, administrators want the error to be presented as a human task in this portal instead of receiving a bunch of e-mails. This needed to be implemented with a minimum of additional (business or process) logic.

To achieve this the following mechanism is used:

• Use Oracle SOA Suite’s Fault Management Framework to redirect (technical) errors to a custom Java class;
• Have the Java class fire an event containing the unique id of the instance using the Event Delivery Network (EDN) or Advanced Queuing (AQ);
• Terminate the composite instance by using the Fault Management Framework and the outcome of the custom Java class;
• Create a single SCA composite to handle all technical errors. This composite subscribes to the event, gathers information on the faulted composite instance, and presents this information as a human task that is assigned to administrators.

This mechanism is shown in the figure below and will be elaborated on in more detail in upcoming blogs.

Logging messages in Oracle SOA Suite 11g using OWSM

This blog is based on a recent OTN forum question and a customer project. In both situations the exact contents of SOAP request and response messages had to be logged, inspected and analyzed.

In one such case policies were applied on services and references within SOA composites to enforce authentication based on WS-Security and add WS-Addressing and timestamp headers. Analyzing interoperability issues is often more problematic when security is involved. Inbound and outbound messages might be encrypted. Besides, probably not that much information will be logged since this poses a possible security threat. You do not want to provide unauthorized persons (possibly hackers) too much information about the protocols, versions, standards and data are expected by your secured service.

A scenario illustrating the need to log SOAP message contents is compliancy testing for Digikoppeling (formerly known as OSB or Overheid Service Bus): a Dutch government standard for exchanging digital messages between government and commercial organizations. Digikoppeling is based on Web Service technology and specifies two sets of standards: WUS (WSDL, UDDI and SOAP) and ebMS. Security is also addressed in these standards. Organizations that want to exchange messages according to the Digikoppeling standard can use a centrally available compliancy test facility. During such compliancy  tests it is essential to be able to inspect SOAP message contents.

So how to do this in Oracle SOA Suite 11g? Well, there are several options.

At first sight you might want to increase the logging level for composites. You can use Enterprise Manager Fusion Control to set the global audit level or configure this per composite. Both can be configured in Enterprise Manager Fusion Control. In the latter case, select the SOA composite in the navigator of Enterprise Manager displaying all composites. Click “Settings” –> “Composite Audit Level” –> “Development” (finest audit level).

Setting SOA composite audit level using Enterprise Manager

However this might not suffice since the exact SOAP message contents (including the SOAP envelope and SOAP headers) are not always logged. In this case, the policies offered by OWSM or Oracle Web Services Manager might help. Especially policies in the management category.
OWSM provides a logging policy that you can attach to your services, components and references. Right-click your SOA composite in Enterprise Manager and select “Policies”. Select “Attach To/Detach From” and select the service, component or reference you want to log messages for. Attach the policy named “oracle/log_policy”. The policy is enabled by default.

Attaching OWSM logging policy to service of SOA composite

Now invoke the composite using a test client. The SOAP request or response message (based on whether you attached the policy to a service or reference) is logged to the following location: [Oracle Home]/user_projects/domains/[SOA Domain]/servers/[SOA Server]/logs/owsm/msglogging/diagnostic.log. Note that several other policies -for example most security type policies- include logging assertions. Policies are composed out of such assertions. If you enable these logging assertions for a policy, SOAP messages will also be logged when these other policies are applied.

An example of a logged SOAP request message (including WS-Security headers) as logged to diagnostic.log looks like the following:

[2010-03-26T09:33:01.243+01:00] [soa_server1] [NOTIFICATION] [] [oracle.wsm.msg.logging] [tid: 16] [userId: <anonymous>] [ecid: 0000IUPa^mx5yW^5xV_AiW1Bf6r800000A,0] [WEBSERVICE_PORT.name: RetrievePatientTreatments_pt] [APP: soa-infra] [WSM_OperationName: process] [J2EE_MODULE.name: fabric] [WEBSERVICE.name: retrievePatientTreatmentsClient_ep] [J2EE_APP.name: soa-infra] [WSM_PolicyVersion: 1] [WSM_LogType: Request] [WSM_RemoteAddress: 10.10.10.10] [WSM_PolicyName: oracle/log_policy] [WSM_ServiceID: soa-infra/InsuranceComposite/retrievePatientTreatmentsClient_ep] [[
<?xml version = ‘1.0′ encoding = ‘UTF-8′?>
<soapenv:Envelope xmlns:book=”http://www.oracle.com/soasuite11g/book” xmlns:soapenv=”http://schemas.xmlsoap.org/soap/envelope/“>
<soapenv:Header>
<wsse:Security xmlns:wsse=”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd” soapenv:mustUnderstand=”1″>
<wsse:UsernameToken xmlns:wsu=”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd” wsu:Id=”UsernameToken-1″>
<wsse:Username>weblogic</wsse:Username>
<wsse:Password Type=”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText”>weblogic1</wsse:Password>
<wsse:Nonce EncodingType=”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary”>5oXaNox3LMvKnDBB/oGeXg==</wsse:Nonce>
<wsu:Created>2010-01-02T21:18:53.957Z</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>
</soapenv:Header>
<soapenv:Body>
<approach:insuranceCompanyRequest>…</approach:insuranceCompanyRequest>
</soapenv:Body>
</soapenv:Envelope>

In case messages are exchanged between composites running on the same SOA Suite Server, local optimization might be applied. That means the SOAP stack is bypassed and Java (RMI) is used for service invocation. Some OWSM policies will not be enforced when local optimization is used. See Vikas Jain’s weblog for information.

It’s often in the process, not always in the tool

Machiel Groeneveld and Mary Beijleveld  just finished a project at an educational institution. Schools often work in close cooperation with other schools and are usually part of a larger partnership with various schools in there municipality. Part of the cooperation and collaboration is regarding allocation of types of education and schools. This school for predominantly HAVO / VWO education (ca. 1500 students) and a smaller part consisting of VMBO vocational education (ca. 500) has more than 200 people working there. Personnel consists of teachers, deans, staff, facilities, and a few people minding all IT systems.

Approach was asked to advice the school on how to make best use of ICT resources for creating schedules for students and teachers and the formation of the teachers workforce.

In summary, the experienced problems were  late completion of grids/schedules and therefore late insight on how many and for which subjects to hire teachers. This also depends on prognosis  on how many new students registered, how many student proceed to next school year and school type and has great consequences on school budget . Our first view on the case was that content of programs and applications were not synchronized, information wasn’t up to date and necessary information not congruent.

In order to give a practical and good advice we first had to identify the most crucial problems regarding process, what IT resources where available, in what manner IT resources were used, and what priority solving a problem has to the school stakeholders. To find out what bothers most, takes the most time, how many and which stakeholders are involved, we held interviews with all stakeholders.

Machiel has in-depth knowledge and experience on how IT systems work and knowledge on some pretty important lean practices. Mary is expert in BPM methods &  techniques and has experience in solving organizational issues. Working together enabled us to take a multidisciplinary view at the problem and to assemble the best and most valuable advice for this customer.

In the interviews we walked, step by step, thru the whole end-to-end process to see where bottlenecks occur, where transfers to other roles were necessary (or not) and which IT resources were used to support stakeholders and process. We didn’t use a sophisticated tool, just rounded A5 papers and pencils or whiteboard and markers to make process visible.

In short, problems were caused by low assessable homemade systems (access dbase, spreadsheets), synchronization of information between homemade systems & commercial products and between Do-It-Yourself systems themselves. Furthermore, process was not ranged optimally and errors easily occurred. Limited accessibility to the content of programs and applications (for example the scheduling makers) and various officials at different times and places making changes in program content.

For quick wins Approach, amongst others,  advised on making clear decisions on moments and responsibilities within the end-to-end process, communication and governance on decisions  & appointments  and improvement on the use of (DIY) systems by supplying a ‘howto’.  For long term alleviation, Approach suggested sensitive communication to understand who needs what information in which step, where an inaccuracy cascades in multiple errors further on in process and disciplined actions for controllability and management.  And we recommend on further research on functionality, integration capabilities and interconnectedness of their existing or future IT resources.

When we presented our report, stakeholders said they didn’t expect to get such advice. They expected something like: ‘get rid of your IT resources, DIY systems and buy ‘this’ one’.  At the end of the presentation session we helped stakeholders to realize about their own responsibilities in the process and to decide about the next approach. The school leaders expressed they were very pleased.

Some tips & tricks on migrating SOA Suite 10g to 11g – Part 2

This blog contains some experiences taken from our migration from SOA Suite 10g to SOA Suite 11g. The previous one was about custom XSLT functions, sensors, composite instance tracking, and Domain Value Maps (DVM). This entry is about using Oracle Internet Directory (OID) 10g as identity provider for SOA Suite 11g.

Integrating OID 10g with SOA Suite 11g
Using OID 10g as identity- and access provider in SOA Suite 10g wasn’t entirely trivial. After applying the steps as documented in Oracle BPEL Process Manager Administrator’s Guide 10g you needed to perform some additional configuration steps that could be somewhat tricky at first. Jaap Poot has some great blogs on this.

Oracle Service Bus article on OTN

The Oracle Service Bus article Eric Elzinga and I wrote is published on Oracle Technology Network (OTN).

The article is aimed at developers and architects who are familiar with Oracle Enterprise Service Bus (OESB) and are (fairly) new to Oracle Service Bus (OSB). The tutorials in this article highlight differences between these two products. The tutorials are based on a workshop in the WAAI community; a collaboration of Dutch consultancies (Whitehorses, Approach, AMIS, and IT-Eye). The goal of the WAAI collaboration is to share, bundle, and expand knowledge on the recent Fusion Middleware 11g release.

Governing events and architect anti-patterns

As the name suggests, SOA is all about services. What about events? In the past, several SOA-efforts tended to neglect events; ultimately causing SOA not to deliver on its full potential or fail altogether. So SOA-practitioners evangelized the use of events. And of course we as IT-industry came up with new terminology to emphasize this: EDA, SOA 2.0, and event-driven SOA to name a few.

This blog is not about promoting events since its importance is (hopefully!) recognized and events are mainstream in nowadays SOA-initiatives. If not, I encourage you to read this blog that explains why events are important from both business and technical perspective. There can be no real SOA without events. Events are just as important as services!

Presentations Oracle OpenWorld 2009

Oracle OpenWorld and Oracle Develop 2009: It’s a Wrap! Just like last year an awesome event! Read about some of the highlights and experiences in this previous blog.

Lonneke Dikmans and I presented the following two sessions on Oracle OpenWorld 2009 that can be viewed here:

Best practices 4 – Security and Identity Management

This is the fourth blog in a series of BPM and SOA best-practices. The previous blog in this series was on Oracle ESB and Mediator. This blog will discuss security and identity management in an SOA-environment.