The high concept of SOA (service-oriented architecture) continues to enthrall IT. Yet SOA's promise of universal application integration is vague at best, confounding anyone who takes a closer look. Such scrutiny reveals major gaps -- in reliability, security, orchestration, legacy support, and semantics. Peter Underwood, vice president of software development at brokerage firm Wall Street Access, says his team has had to do some serious thinking up front before planning SOA integration.
"You begin with the idea that (SOA) is bigger than a bread box. In other words, it's just a framework," Underwood says. Although SOA "has taken on a life of its own because of Web services" standards, Underwood believes a significant gap remains between Web services' potential and its current capabilities.
Execs are happy to use Web services for simple needs, such as feeding information to Web-based portals. But complex, mission-critical jobs are another story -- and may demand Web services standards that are still under development. So when is a Web services SOA strategy advisable, and when is good old EAI better? It all depends on what you are trying to do and which gap in Web services' capabilities you encounter.
The need for highly reliable, asynchronous messaging may be the most difficult to meet, at least in the short term. Aiaz Kazi, general manager of business integration at EAI stalwart Tibco Software Inc., calls this kind of messaging "critical to enterprise-quality integration."
Sam Boonin, vice president of marketing at Web services infrastructure vendor Blue Titan Software Inc., says the need for reliability is similar to that which "we're used to discussing in other computing paradigms. SOA is not quite ready for the utmost transactional reliability -- nonrepudiation, once-and-only-once delivery, and rollback -- but it's only a matter of time until the standards and implementations mature to meet that requirement."
In fact, several draft Web services specifications already address issues in mission-critical and lengthy processes. WS-ReliableMessaging, for example, is designed to guarantee that a SOAP message arrives at its destination. WS-AtomicTransaction, WS-Eventing, and several other proposed specifications would define ways of handling complex, stateful, and long-running business transactions. But unlike many security-related protocols, widespread use of reliability standards such as these have yet to be realized.
Until then, says Chris Crowhurst, vice president of enterprise architecture at Thomson Prometric, a provider of computer-based testing and assessment services, "Reliable messaging (for Web services) is quite a burden. But at the end of the day, applications just need to build around it" because of the benefits of the interoperability Web services provides.
For now, "building around it" means coding applications to anticipate and to accommodate error conditions. It also means buttressing point-to-point SOAP interactions with an intermediary -- such as a Web services management broker -- that provides a standardized layer of abstraction. Available from independent players such as Actional Corp., AmberPoint Inc., and Blue Titan, these products enable managers to provide fail-overs and upgrades to software endpoints with minimal interruption to the production systems. (Useful Web services management must work across a range of platforms, which explains the absence of similar solutions from such major vendors as BEA Systems Inc., IBM Corp., and Microsoft Corp.)
On the other hand, as Dan Foody, CTO of Web services management vendor Actional, notes, "Not every problem requires the same kind of reliability." Those that must be ironclad tend to be asynchronous, long-running transactions with many interdependencies such as complex financial transactions. For less demanding jobs, SOAP over HTTP works fairly reliably -- particularly with simple, synchronous interactions.
Rajan Jena, an enterprise architect at Bristol-Myers Squibb Co. subsidiary Oncology Therapeutics Network, uses both conventional messaging-oriented middleware and Web services in his company's integration infrastructure. He says that messaging solutions are appropriate when transaction volume is high and is transmitted in batches. On the other hand, Web services are best "when volume is low -- but it has to be totally real-time."
Authorization, authentication, and encryption raise a serious red flag for IT managers contemplating Web services-based integration. Traditionally, access control has been a matter of requiring a log-in and authentication. In the distributed Web services world, where components of one application might easily go off and talk to components that live in different domains, keeping disparate but interconnected systems secure is a far more complicated problem.
As with reliable messaging, a bevy of standards for Web services-style interactions have been proposed. Two are particularly important and are being implemented widely: WS-Security and SAML. The former describes a highly extensible framework for itemizing various facets of a system's security capabilities, whereas the latter defines a standard process of transmitting assertions to facilitate SSO (single sign-on) models of authentication.
For enterprise SOAs, analyst Phil Wainewright of the lively Loosely Coupled discussion site (infoworld.com/1859) singles out a third, though not yet mature, proposal. WS-Policy would provide a means for different systems to declare what sorts of security mechanisms they require before accepting connections. "Without it, your ability to loosely couple across domains (would be) severely limited," Wainewright says.
Although no proposed standard meets the Web services security challenge by itself, as a group they provide a foundation on which to plan an ongoing strategy. Indeed, the Web Services Interoperability (WS-I) vendor and user group recently ratified portions of WS-Security for its Basic Security Profile, a collection of best practices designed to ensure interoperability. The organization is expected to add support for SAML and other standards in the near future.
In the short term, a relatively practical way to secure Web services is to employ the the techniques used to secure standard Web-based applications: transport-level mechanisms such as SSL.
One advantage of SSL is its support for two-way authentication in the form of client/server certificates, but the ongoing maintenance of a certificate infrastructure for a large number of clients can be a challenge. A complementary option that is available today is to employ generalized XML-savvy firewalls -- such as those from DataPower Technology Inc., Forum Systems Inc., Reactivity Inc., and Westbridge Technology Inc. -- and XML digital signatures to provide an additional measure of prevention for Web services network traffic.
After a distributed security architecture is in place, IT managers should find a silver lining in the security cloud. Ongoing maintenance of application security has often fallen on the shoulders of programmers -- a poor use of their skills and a potential security hazard. With Web services-based security, it becomes much more feasible to restrict that responsibility to authorized operations staff alone.
Oncology Therapeutics Network's Jena describes this security approach as critical in his company's transition to an SOA for its CRM and order management systems. "That means we can implement (security) from a policy perspective, as opposed to developers actually coding it" into the applications, he says. "We see that as a major advantage. It actually makes the applications more secure."
The coordination of distributed software components for the purpose of creating meaningful business processes is at once the most complex and the best-suited to service-oriented styles of integration. The reason is clear: In the simplest of terms, applications built on SOAs are designed to be pulled apart and reassembled as needed. As the backbone of today's BPM solutions, "orchestration," as it is called, enables IT managers to string together new meta-applications from the functionality of packaged or homegrown applications already in place.
Blue Titan's Boonin says this capability is important to many adopters of SOA integration strategies. "(They) are building composite applications on top of (SOA infrastructure) and orchestrating services into reusable business processes," Boonin says. On the surface, assembling these pieces to guide the flow of a business transaction is a straightforward task. After all, the primary stumbling block for BPM solutions has been the monolithic nature of software. Web services, on the other hand, break software into components, each of which relating to a single business function.
The biggest challenge isn't to create the modular applications but to change the way those systems represent the data they process. Thomson Prometric's Crowhurst counsels "identifying a schema at the heart of the business," rather than designing process flows around individual systems and their data records in isolation. In other words, it's best to think about the output of business processes as documents that contain answers to each major question along the way. XML, the foundational element of Web services, enables this conception and makes it happen. "Thomson Corporation is XML-obsessed," Crowhurst says.
Several standards for orchestration and BPM have been proposed. One of these, BPEL (Business Process Execution Language), has broad industry support and even appears in a number of BPM products -- although everyone concedes that the robust version, BPEL 2.0, is approximately 18 months from completion. Complementing BPEL, WS-AtomicTransaction and WS-CAF (Composite Application Framework) are designed to facilitate the complex transactions that make up long-running and stateful business processes.
BPM pure-plays, traditional EAI companies, and the major platform vendors all see promise in the ability to visualize, execute, and monitor processes using this new, standards-based form of BPM. Enterprise application vendors such as SAP AG and Oracle Corp., which bought the BPM pure-play Collaxa Inc. this year, are also on board. SAP is taking a radical approach, actually redesigning its software to incorporate its own version of BPM-focused middleware, dubbed NetWeaver. Finally, XML-centric data stores specifically designed for SOAs, such as Blue Titan's Data Director and Software AG's Tamino, will represent an increasingly important aspect of addressing business process schema.
The software solutions used to connect applications with legacy systems are not unlike the kits that world travelers carry in their briefcases. Just as the right sort of dongle allows an American visitor in Greece to plug a laptop power cord into the wall, a specialized application adapter allows one system to pull data from another.
And just as each country seems to have its own standard for power outlets, each legacy system requires its own adapter. For years, EAI vendors emphasized their portfolios of legacy application adapters as key differentiators -- and as sources of revenue.
Fortunately, as standards such as Web services increasingly dominate interoperability, the cost of simple connectivity has fallen. And third-party application adapter specialists such as Attunity and iWay have increased competition and have vastly expanded the number of connectivity options available today.
A sometimes overlooked benefit of legacy application adapters is that they mask the complexity and obtuseness of many proprietary APIs. The role of a good adapter is not unlike that of a well-composed Web service: It provides a layer of abstraction that insulates the rest of the application infrastructure from all sorts of messiness. Some vendors, such as Software AG, have specialized in the "semantic integration" of legacy applications to XML-based integration backbones.
Packaged applications from companies such as Oracle and SAP already provide some degree of support for standards-based connectivity, typically by wrapping a proprietary API, such as SAP's Business API, with a SOAP interface. As application vendors make use of Web services and add middlewarelike functions to their own products, the standard method of integrating packaged applications will come to reflect the best practices of SOA.
Even so, dealing with expensive, proprietary application adapters will continue to be the only option for connecting many truly archaic systems to a common integration framework.
There is one silver lining for IT shops with mainframes from companies -- such as IBM -- that have aggressively ported basic Web services stacks to their antique equipment. Ironically, their rigid APIs can be far simpler to expose through SOAP than untangling the mess of APIs found in many client/server systems.
Defining the business meaning of transactions and data is the most intractable issue that IT managers face. Although the semantics challenge predates Web services -- vertical industries have been developing their own XML schema for years, for example -- SOAs bring semantics to the fore. In fact, semantic interaction is a central part of well-designed SOAs.
No technology or software product can truly solve the semantic problem. Business and IT managers must ultimately shoulder the hard, painful work of defining and implementing functions and data models for industry- and function-specific processes. Nonetheless, prebuilt components and battle-tested consulting expertise can simplify many of the challenges.
EAI vendors see their greater experience building integration solutions as a hedge against the commodification that Web services standards portend. Tibco's Kazi puts it bluntly: Software integration "is a party we went to many, many years ago -- we almost started this party." A cursory look at his company's Web site reveals specific solutions for industries ranging from telecommunications to health care to transportation.
As they move to support SOA, packaged application vendors are also touting their experience with specific industries and business processes as an asset. In particular, SAP emphasizes its xApps initiative, which represents a Web services-based implementation of business processes that cross the traditional "silos" of vertical and functional systems.
Increasingly, companies are recognizing the value of XML-based standards for their own industries. For example, the financial services industry has developed FIXML (Financial Information Exchange Markup Language), an interchange protocol for bank transactions; and the accounting industry has proposed XBRL (Extensible Business Reporting Language) for describing and auditing general ledger-style records. High-tech manufacturers continue to use the granddaddy of XML vocabularies, RosettaNet, to exchange product and component information in their supply chains.
In fact, many observers say that the dynamics of a company's industry may be one of the most important drivers as to when and how a move toward SOA will happen. Bob Sutor, director of WebSphere infrastructure at IBM, notes that industries such as financial services are particularly active adopters of SOA. In addition to technology catalysts such as vertical industry standards, he points to a high number of mergers and acquisitions as a major contributor to this trend. Rather than slowly sorting out and replicating information from one company's systems to another, he says smart adopters should use SOA-style efforts to "actually integrate business units onto the network" quickly and with less disruption than they would otherwise.
In spite of these challenges, most IT managers agree that adopting a service-oriented integration infrastructure is more a question of "when," not "if."
Even so, the old adage, "measure twice, cut once," applies in spades. Without clear business analysis and principles guiding the development, "you wind up recreating interfaces, which is a real nightmare," Wall Street Access' Underwood says.
David Sprott, CEO and principal analyst of research company CBDi, agrees but cautions that the transition to a service-oriented strategy shouldn't be technology-driven. "Using services thinking (helps) to get business and IT on an equal footing. In other words, don't just do services because it's cool. Drive activity on the business requirement and ROI criteria," he says.
Thomson Prometric's Crowhurst echoes that learning how to represent essential business processes as services is critical. "It takes a cultural shift to change how development happens," he says. "Compared to that, the technical challenges are just an intellectual exercise."
Brent Sleeper is a principal at The Stencil Group, a business consulting and advisory services firm.
What about performance?
Critics of Web services and SOA often cite performance as a barrier to adoption. But there's always a speed tradeoff in standardization.
The five challenges highlighted in this article reflect trade-offs intrinsic to the distributed, loosely coupled nature of Web services-based SOAs. But skeptics frequently cite another issue -- performance -- as a particular weakness of the model. This criticism generally has two parts: the distributed nature of SOAs and the overhead of Web services protocols.
Any distributed system performs slower than a self-contained one, simply because the network becomes the limiting factor. And of course, some applications can't tolerate network-incurred delays. But that axiom isn't unique to Web services or SOA. The benefits of flexibility and access gained by keeping systems close to the units of work they represent -- and connecting those systems to others across an organization or across the Internet -- far outweigh the performance hit introduced by the network.
More germane is the relative weight of XML transactions when compared with binary ones. The effort it takes for a system such as a Java application server to serialize and deserialize SOAP calls into internal objects is estimated to be 10 times higher than that required for native approaches such as Remote Method Invocation. This sounds meaningful until you consider the lessons learned from the Web itself. Like HTML, XML is a very wordy approach to describing information. Yet it is precisely that readability and extensibility that compensates for the performance impact, while Moore's Law -- along with specialized devices such as XML accelerators -- moves ever closer to canceling out the performance implications across time.
In the end, neither Web services advocates nor critics hold every card. It's a question of using the right tool for the right job. High-volume transactions will continue to be addressed by proprietary, binary middleware solutions for some time. But applications that require less frequent but much more flexible interactions will be served, increasingly, by Web services. -- InfoWorld (US)
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.