Rourke on SOA

People have been using the term ESB — short for Enterprise Service Bus — for quite a few years now. Despite that, there continues to be enormous confusion over what the term means. This confusion stems from the fact that people have overloaded the term ESB and are using it to mean two different things. When people talk about ESBs, they are either talking about ESB products (ESBp) or ESB implementations (ESBi). No one can sell you an ESB implementation, just as no one can sell you an SOA. However, you can and should buy products that help to implement this design pattern.

ESB implementations (ESBis) are manifestations of an enterprise design pattern intended to simplify the interconnection of services in a given environment. Most enterprise architectures today include some form of ESBi. Many SOA reference architectures include some form of ESBi. A few years ago ESBi was purely theoretical, but today there are mature implementations that cover the complexity gamut, from simple to very complex. I’d like to examine the different levels of ESBi in the future.

ESB products (ESBp) are designed to help companies build ESBi. The ESB market is mature and the definition of what makes an ESB product (ESBp) has been made clear over the course of 2006. This year has seen every major SOA vendor release of refine their ESB offering or offerings. Most of the analysts have released ESBp reports that score or compare the products. A number of magazines, including Network Computing and InfoWorld, have released ESBp round-ups and reviews. Folks, as 2006 ends we finally have a clear definition of what an ESB product is.

First, lets eliminate some of the confusion the acronym causes: not all ESBp offerings include a “bus” or messaging technology. Most companies already have some form of messaging technology and are best served by using an ESBp that does not tie them to a specific messaging product, but instead allows them to build out their ESBi using the messaging technology they already have. If you don’t have an enterprise-grade messaging technology already, then either purchase an ESBp that includes messaging, or purchase a best of breed messaging product in addition to the ESBp you select. Do not rely on a sub-par messaging technology simply because it came bundled with your ESBp.

At a minimum, an ESBp must provide for mediation on several levels, as listed below. This allows you to use that ESBp to onboard services to your ESBi. To facilitate collaboration and to facilitate re-use of the services you onboard, a good ESBp must integrate tightly with the version control systems and registry products in your environment. In most cases, registry support just means UDDI support. In order to deal with routing, cross referencing, and exception handling, a good ESBp will also have lightweight process support.

• transport level mediation, at least:
  • HTTP
  • JMS
• message format mediation, at least:
  • SOAP
  • POX (plain old XML)
• security mediation (in the form of SSL and WS-Security)
• message format mediation (in the form of XSLT/XPath)

Performance, scalability, and fault-tolerance are assumed. An ESBp acts as part of your communications infrastructure, and any downtime will translate into service downtime. Any latency issues will translate into service latency issues.

Even an entry-level ESBp must include a complete SOAP stack with support for all flavors of SOAP. If everyone exposed their services via WS-I compliant services there’d be far less need for ESBp’s.

The above requirements are the absolute minimum for a product to be considered an ESBp. In addition, more advanced ESB products (sometimes called ESB Suites or Integration Suites) will also have many of the following features:

• auditing of messages
• logging of messages
• MEP mediation (request reply : pub/sub)
• a rich set of transports
  • TCP
  • MQ-Series
  • TIBCO RV
  • Java RMI
  • SMTP/POP3
  • FTP
  • IIOP
  • COM/DCOM
  • etc
• a rich set of message formats
  • Cobol Copybook
  • EDI
  • fixed format and delimited file formats
  • .Net data marshalling
• rich collection of adapters to provide zero-code access to:
  • application operations
  • application metadata
• advanced monitoring and management
• integrated debuggers

Service composition and service orchestration capabilities are often bundled with ESB Suites and Integration Suites. There are advantages to having these capabilities tightly integrated with your ESBp, as this allows you to easily invoke functionality as well as providing for superior performance due to fewer SOAP hops.

As Rich Seeley points out in this article, the current trend is for people to pile more and more into ESB Suites. Many such suites have grown to include even BPM capabilities. It isn’t clear that buying BPM technology as part of an ESB is a good idea. ESB Suites currently have the best enabling technology for service composition and orchestration, but the best BPM offerings are still the standalone BPM products.

Rich’s article suggests that people might start using the term “fabric” to describe larger ESB Suites which include BPM. Only Webmethods and the Burton Group have begun to do. Expect this to catch on only after Gartner and Forrester change their language.

A number of people have predicted the impending death of ESBs. These predictions don’t make sense to me, given the fact that the world isn’t 100% WS-I compliant SOAP. Frankly, we wouldn’t want the world to be all-SOAPed up because of the complexity and performance issues that would imply. There are quite a few ESB products on the market today because there is a need for such products, and there will continue to be a need for such products for the foreseeable future.