There has always been something alluring about SOA (service oriented architecture), that collection of standards that simplifies and standardises the way various business systems talk to each other. Linking different systems - especially old legacy systems that were never designed to play well with others - has always been a complex, frustrating and fragile process. Little wonder that the ability to standardise those conversations has caught the attention of countless companies that have spent years - and millions of dollars - bogged down in application integration.
Even better, the process of implementing SOA is more understood, and better supported with technology, than ever. Far from just dipping their toes in, large companies are jumping into SOA with both feet. Systems integrator KAZ Group reports handling more than seven SOA projects, each worth more than $1 million, in the past 12 months.
In each case, SOA happened by accident, says the director for applications development and systems integration with KAZ, ¿Ross Magee. "While we didn't start in any of those projects thinking SOA, it turned out to be the answer to those problems," he says. They were "business problems that we had to solve, and in each of the projects, the business problems were thought to be insurmountable. One retailer was looking at trashing its entire IT infrastructure because it didn't have the flexibility to go forward."
Whether producing massive launches or smaller, more exploratory efforts, companies that embrace SOA seem to be enjoying strong success. They're reaping benefits through faster development, easier software upgrades, lower costs and - most important of all - flexible environments that can be more readily expanded to encompass both ageing legacy systems and networks that are yet to be invented.
But there are still many potential problems awaiting those who would pursue SOA. Security, coding efficiency, data management, training, process change and other issues still loom large, each addressed by a variety of solutions but let down by a lack of formal SOA development methodology.
Overall, the SOA situation recalls the early days of enterprise resource planning, which was an equally disruptive technology that forced companies to totally revisit their business and, often, to make dramatic changes to systems that had been in place for years or even decades.
The early days of ERP were marked by a massive series of overblown budgets, failed implementations, and defensive vendors who were often pushing too much technology and too little process. These days, vendors offer ERP for small- and medium-sized businesses and enterprise transformation is now a matter of incremental change.
SOA hasn't matured as much as ERP, in that there are still varying definitions of what it entails and how it should be implemented. As with ERP, however, a single truth remains: if it's done right, at both process and technology levels, SOA can work wonders.
Automotive distributor Nissan Australia knows all about the value of SOA. It has spent much of the past two years refining its approach to core systems and using SOA as a catalyst for significant change within them. Nissan embraced SOA as it began looking for ways to bring its legacy, green-screen, mainframe-based Software AG Natural and Adabas application environment into the web age.
The old systems "still provided the functionality that we needed, and we were still modifying the systems to cater for new business processes," says the national manager of technical services for Nissan Australia, Steve Hogan. "But looking over the horizon a bit, in the new integrated and modernised world, we wouldn't be in a position to [provide] what the business was demanding. It was a very clear realisation that the mainframe must be integrated somehow."
Instead of removing and replacing the old environment, which would have forced the company's entire IT and development team to learn completely new skills, Nissan turned to increasingly SOA-aware Software AG to help it introduce software wrappers that would guide the system into the modern age. Using this approach, SOA could be introduced, along with linkages into newer systems and a new interface into the mainframe, without forcing the company to start from scratch.
The development team started small.
Workers built a few test services to figure out how the SOA environment might be shaped. Now, 18 months on, the company is using more than 100 services and adding more all the time.
"We've arrived at the stage where we feel pretty comfortable with it," says Nissan e-business manager Mikhail Anossovitch. "There was no big bang project as such at the beginning, but each subsequent project was more and more challenging; we're now in a position where there is nothing that the business asks that cannot be delivered.
As with all technology, starting small and growing steadily proved to be the right approach for Nissan. Today's projects are more ambitious than their predecessors; one current large application provides a single interface combining a range of functions for Nissan dealers. However, Hogan makes the point that the company's SOA progress is guided not by business or technical goals, but by paying careful attention to human resources and the training needed to get developers up to speed.
The company hasn't developed a formal methodology for building SOA elements but the cumulative knowledge of its employees has smoothed out the initial bumps and paved the way for far easier, more consistent implementation than was possible in the past. Hogan concedes, however, that SOA governance may become an issue in the long term, as the number of active services continues to grow rapidly.
Business users "are continually asking for more, and it takes true process improvement for them to deliver that," he says. "It's going to be an evolutionary thing, and we are going to continue."
Not SOA fast
If it is implemented properly, SOA can be a lifesaver, enabling a flexible and capable architecture. But if done poorly, it can be a train wreck - and yet another hurdle in the road to the totally flexible enterprise.
Major systems integrators, who are the first points of call for most companies looking at the benefits of SOA, have seen it all before. Consider, for example, the company that successfully implemented SOA within its divisions but forgot to co-ordinate the efforts, so that each division ended up with its own very effective, very different SOA.
"Because nobody took a co-ordinated approach to it, they had great SOA internally and nothing at an organisational level," recalls the technical lead for SOA with Hewlett-Packard (HP), Raymond Lambie, who has spent the past several years on the ground helping customers work out issues the new technology introduces. "Eventually, they had a bun fight about whose standard to use and chose to apply another, completely different set of standards."
Those standards don't relate to the basic building blocks of SOA - extensible markup language (XML), simple object access protocol (SOAP), and others - but rather to the actual processes necessary to introduce SOA into the environment. These include everything from how SOA skills are gained and how project managers should manage risk in a heavily interdependent environment, to how to keep business managers' expectations realistic despite their eagerness, and so on.
Process change is also necessary to address the ancillary issues that SOA introduces. Security, for example, must be addressed, through technological means and through processes that identify the types of breaches SOAs may enable, and specify the remedy when there is a problem.
"XML is exciting technology, but its power provides new threats that have to be catered for," KAZ's Magee explains. "Most clients, we have found, are not quite prepared for an XML world, in terms of security. While it's a fantastic interoperability standard, it kills performance if you don't architect your solution appropriately."
These performance hits are due largely to the uncompressed and often voluminous format of most XML data. And they compound potential security problems. The result: SOA adopters need to tread ever more carefully when it comes to ensuring they can accommodate the unique aspects of SOA development.
After all, the SOA construct is built around one large, highly integrated conglomeration of services and applications, uniting form and function in a new way. Although the end result is different, the architectural philosophy in play here isn't that far removed from ERP, which consolidated many diverse types of business information for the first time and ignited the imagination with the possibilities it presented.
As ERP customers and integrators gradually wrestled their newly found technological future to the ground, the market underwent a significant shift. Customers stopped blowing their budgets, wholesale failures declined, vendor upgrades focused more on incremental improvements and less comprehensive all-in-one solutions gradually brought the technology down to the level of the small business.
A long way to go
Based on this aspect alone, SOA still has a long way to go before it can be mature enough for every company to implement. So if you're a small business with development know-how, it's more than possible to dabble in SOA, but you may soon be in over your head if you're hoping to do something more significant without major preparation and a formal game plan.
"Methodology and architecture take some time to internalise," Magee says. "A lot of these projects aren't long-term problems to fix; they're acute pain points at the moment. The rise of standards is very compelling, but people get a bit scared of just how much work is required."
In the ERP world, such game plans are widely available. So far, however, the market has lacked a counterpart in the SOA world; without clear definition of what an SOA implementation involves, it has been hard to formalise the methods for putting one in place.
Even that situation looks set to change, however, as the market warms to a formal set of procedural guidelines for SOA implementations: The Open Group Service Integration Maturity Model (at www.opengroup.org/projects/osimm/), has the support of industry giants IBM, BEA Systems, Capgemini, EDS and HP. It was launched by The Open Group in February 2007 in an effort to create a realistic set of milestones that companies can use to judge their progress towards SOA.
OSIMM includes seven levels of maturity: silo, integrated, componentised, simple services, composite services, virtualised services and dynamically reconfigurable services. Each level sets specific qualitative descriptors in seven areas, such as governance, applications, architecture, information, management and so on.
The distribution of milestones between levels comes from the vendors' experience with early SOA adoption, and things like poor control mechanisms that were problems in the past. "The procedural aspects of deploying the services are key," HP's Lambie says.
Success, he continues, requires "giving due consideration to expose the practices you need, and the understanding of the relationships and dependencies that exist: the points of control, which business unit uses it, who has control over its deployment, and who manages the operational state. That's where most organisations fall down, because they either don't know or [haven't fully considered]" the full spectrum of controls that is necessary.
Just as many companies have rushed to certify their development and support using the Capability Maturity Model (CMM) paradigm, OSIMM will no doubt accelerate the efforts of many companies that are keen to get on with SOA. It's important to remember that in the OSIMM model, an organisation hasn't entered the SOA realm until it reaches level four - simple services - as judged by internal or external assessment, in the paradigm's milestones.
The OSIMM effort is arguably a step in the right direction but it doesn't seem to have gained significant traction yet. The documents on its website haven't been updated since the launch at the beginning of 2007, and there doesn't seem to have been any ratification of the standard, as was initially flagged in April.
Whether this delay is due to perceived immaturity in the OSIMM model or ongoing conflict over its correct composition, or is happening simply because most customers are still fumbling in the dark for the SOA light switch, isn't clear. What is clear is that such efforts will be essential in helping vendors and customers speak a common language discussing SOA expectations.
This could be the last crucial piece before SOA hits the mainstream, says ¿Ross Moodley, SOA leader for ¿IBM Global Business Services, which seeded the first draft of OSIMM, based on its own experiences and SOA methodologies.
"Most organisations tend to be more advanced in the applications and architecture area, and less advanced in the business amalgamation and optimisation areas," he explains. "The real value of SOA, as we see it, is absolutely at the business level.
If you have the right business model defined, and identify the business services that are going to make a significant difference, you're going to be getting the biggest return on your SOA."
Methods slowly emerging
Moodley knows all about the pitfalls and opportunities SOA poses.
As a key figure in the systems for people consolidation project that the Department of Immigration and Citizenship is running, he spent nine months commuting from Sydney to Canberra.
His efforts have helped steer a development team of more than 200 IBM global business services staff - fully one-third of its entire regional developer force - through a year-long project valued at $496 million. The project is built completely around an SOA core and stands as one of the world's biggest SOA implementations. That's a long way to come for a once-nebulous development concept.
Tentative SOA metrics are emerging and vendors are gradually agreeing on a more consistent set of standards, so the technology's best days are yet to come. High-profile SOA efforts from SAP and Oracle take an everything-but-the-kitchen-sink approach, but smaller vendors are getting their acts together, too.
"SOA is about opening the product up, so people can call the various componentry that makes it up in a very much industry standard fashion," says Adrian Di Marco, executive chairman with ERP vendor ¿TechnologyOne. "SOA promotes the customisation process. But while it's something people are talking about, not a lot of people are doing it yet."
That's likely to change. As with ERP, business leaders who have been enlightened about the possibilities of SOA quickly come to love it, but it's crucial that technology and business leaders collaborate to develop a smooth methodology that will get them where they want to go.
Technical leaders can make SOA happen, as they did for ERP, but the biggest mistake business officials can make is to relegate the technology to the technologists.
In the end, SOA is all about solving problems for business. Once the terms of reference are clarified and a clear implementation path is established,
the sky's the limit.
© Fairfax Business Media
Join the CIO New Zealand group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.