Posts Tagged ‘ESB’

Best practices 3 – Oracle ESB and Mediator

This is the third post in our SOA and BPM best practices series. This blog provides best practices for Oracle ESB (Oracle Fusion Middleware 10g) and its successor (when it concerns routing and transformation): the mediator component in SCA (Oracle Fusion Middleware 11g). The previous blog in this series is about Web Services best practices.

Passive adapters in Oracle ESB that won’t be activated

Configuring SOA Suite 10g for high availability (HA) isn’t the most easy thing to do. Several administrators I spoke with and worked with in projects brought this up. I really hope that FMW 11g -besides all the new functionality, enhancements and support for new standards such as SCA- also makes things like HA easier to configure.

One particular issue we recently ran into in one of our projects has to do with the use of non-concurrent adapters in Oracle ESB when upgrading our clustered environment from 10.1.3.3 to 10.1.3.4. Non-concurrent (or singleton) adapters are adapters that cannot run in an active-active configuration since the underlying infrastructure does not provide a good locking mechanism. Examples are file and FTP adapters. JMS and database adapters on the other hand support concurrency. For non-concurrent adapters you have to ensure that there is only one adapter instance active at runtime. Otherwise you could have two active file adapters both reading the same file, starting two ESB flows instead of one. Futhermore, you want to have fail-over. If the ESB RT (runtime) node on which the active file adapter is running (or adapter itself) fails, the passive adapter on another ESB RT node should be activated. In earlier SOA Suite 10g releases you had to install and configure a separate ESB RT for this (ESB Singleton) and deploy non-concurrent adapters to this separate node. Real overkill. Fortunately, in later versions you could deploy non-concurrent adapters to the existing ESB RT’s and configure these adapters in an active-passive configuration by setting the clusterGroupId property. The jGroups protocol is then used so that only one instance of all adapters that have the same clusterGroupId value will be activated.

When we upgraded to SOA Suite 10.1.3.4 none of our file adapters in the acceptance environment was activated anymore! After some investigation it seemed that ESB 10.1.3.4 uses its own jGroups configuration instead of the jGroups configuration as specified in the global jgroups-protocol.xml file (as was the case for ESB 10.1.3.3). That isn’t a problem by default. However, in our case both our test and acceptance environment are clustered and both run in the same network. The internal jGroups configuration of both test and acceptance by default probably use the same ip and subnet addresses. Meaning all adapters of all ESB projects in the same network with the same clusterGroupId are all put in the same active-passive configuration. For ESB project “A” only one file adapter instance for test was active, the same file adapters for ESB project “A” for acceptance were all in passive mode. Luckily you can specify the useJgroupConfigFile property for an ESB endpoint and set it to true to enforce using the jgroups-protocol.xml configuration file; as was the case in ESB 10.1.3.3. Then configure a different ip and subnet address combination for test and acceptance. That way the non-concurrent adapters in the same ESB projects but in different environments are separated when they have the same clusterGroupId. Another workaround would be to include the environment name in the clusterGroupId value, e.g. MY_ESB_TEST_ID and MY_ESB_ACCEPTANCE_ID.

Custom adapter article on OTN

A while ago someone asked me if you could create your own adapter and use it from Oracle SOA Suite. More specifically if you could create an inbound e-mail adapter (not available out-of-the-box in SOA Suite) that polls for new mail messages and use it as activation agent for starting a BPEL process or ESB flow.

I knew this was possible -at least the possibility was documented- and searched for a how-to. I couldn’t find one explaining all steps involved. However, I did find lots of questions on the OTN forums asking how to achieve this. I thought it would be nice to write an article about it after I got it working. It’s published on OTN.

The article includes a step-by-step tutorial for building the adapter and plugging it into Oracle SOA Suite components such as BPEL PM and ESB. The article also briefly discusses adapter support, offerings, and convergence in future Oracle Fusion Middleware releases that incorporate former BEA products.

Drop me a line if you’re interested in an example for using a outbound adapter instead of an inbound one. I’ll see if I can add such an example in the future.

Master/detail inserts and native sequencing in Oracle ESB

The SOA Suite forum on Oracle Technology Network (OTN) contains several posts on the use and configuration of the Database Adapter. This adapter can be used to perform database-related operations such as inserts, polling, invocation of PL/SQL, and so on in your SOA-environment. Some of these forum questions relate to inserting XML data into master/detail tables and how to use native database sequences to populate primary keys and automatically update associated foreign keys. I wrote an article Invoice Processing in a Service-Oriented Environment that contains a step by step tutorial on how to achieve this.

Database adapters, TopLink and Transformations

In one of our current customer SOA projects we’re using Oracle Enterprise Service Bus (ESB) to implement and expose services. One of these services involves transformation of inbound data and persisting this data in a database. We’re using XSL to transform the inbound data and Oracle Application Server’s database adapter to persist data into a relational database. JDeveloper, Oracle’s IDE, provides wizards to configure database adapters. TopLink (an object-relational mapping framework) mappings are generated as a result of this configuration. These mappings are XML files containing metadata such as the structure and format of the database tables. In our ESB flow data is transformed into nested XML format, which is persisted in multiple master-detail tables.

This week -after a modification to the database adapter configuration- data was still persisted, but lots of database records that were previously populated, were suddenly empty. The ESB console indicated all instances were valid and debugging the ESB flows showed that the input XML consumed by the database adapter was unchanged. After some more investigation it turned out that the order of tables in some of the generated TopLink mapping files (OurService_table.xsd, OurService_toplink_mappings.xml and OurService.RootTable.ClassDescriptor.xml) had altered; e.g. instead of table 1, table 2, table 3, the order of tables was table 1, table 3, table 2. The transformation activity still generated input XML data as table 1, table 2, table 3. Synching the order of tables in the transformation and mappings files resolved this issue. This means that runtime, the database adapter strictly follows the order of tables indicated in the TopLink mappings. You can argue whether this is too strict or not, especially since the same tools are very “tolerant” of other faults, omissions, validation, etc. Well, at least you would like to see some kind of error when the XML input format does not conform to the TopLink mapping definition.

Anyway, this was a tricky issue since the input XML seemed valid, the ESB console indicated all instances were valid, no error was logged and some data (but not all!) was persisted in the database.