Top tips to measure the success of SOA installations

Bv Max Cooter

Critics argue that service oriented architectures (SOA) have failed to deliver value. But the real problem might just be that most firms fail to properly measure the costs and benefits of their SOA projects.

While service oriented architectures (SOA) have garnered plenty of headlines over the past few years, some question whether the technology has made the headway that was expected of it. Indeed, earlier this year, analyst firm Burton Group announced that the SOA experiment had failed, as SOA had not delivered all the promised benefits. In fact, Burton said, SOA had delayed projects and increased costs.

An SOA is a software architecture used to build and maintain applications in an enterprise. Rather than designing applications from the ground up, an SOA allows developers to reuse pieces of code packaged as a service and share them between applications and company departments. This lets developers combine resources from all over the company, which typically yields cost savings, while better aligning IT services with business processes. For these reasons, the SOA has been hailed as a major technology breakthrough, but at the same time, it requires vast changes in corporate policies and investments in new software products to manage the architectures. Since last year, SOA is said to have largely fallen off the tech radar.
While most observers said that the Burton Group had overstated the problems and that SOA still generates plenty of interest, the Burton study did put its finger on an important issue when it comes to SOA: how should one measure a project’s success? And what metrics should an organisation use for this process?

SOA guru David Linthicum, author of the forthcoming book Cloud Computing and SOA Convergence in Your Enterprise, thinks that the Burton report has been misinterpreted and made some important statements about SOA. What Burton highlighted was the difficulty of assessing the success of a project, he says.

For Linthicum, there are three basic approaches to measure a project’s success. An enterprise can use one of the methodologies from the likes of the Open Group, that follow a standards-based assessment approach; they can use methodologies from a vendor; or a firm can use its own in-house devised methodology. Linthicum says that the best approach is one that marries one of the more rigid methodologies with an enterprise’s own customisation.
He also points out the SOA is a process – not something tacked on to an existing way of doing business. “SOA is something you do, not something you buy,” he says. “Anyone treating it like something you can buy is doomed to failure.”

One of the reasons for the difficulty in assessing a project’s worth is that SOA is based on a variety of factors: there’s both an IT element – the overhaul of an organisation’s entire computing system – and a business element. Such disruption has to be driven by a business case, and it is this that needs to be defined from the outset.

IBM is a major player in the SOA market. The company’s SOA business leader, Gary Gomersall, argues that a high level of co-operation is essential to ensure the success of an SOA installation, while the business case has the highest priority. “I can’t emphasise strongly enough how important it is to develop SOA close to the business process.” But, he says, the fact that business people and IT staff have different priorities and talk a different language should not necessarily be a barrier to success. “Once the IT guys are talking to the business guys, they between them can often find short cuts that give real value and save a lot of complexity later on.”

Gomersall says that after having determined objectives, the next step is to draw up a two-dimensional matrix –based on a series of what he calls Key Agility Indicators, which would include features such as ‘time to market for product’ or ‘time to new channel’, and a series of Key Performance Indicators, those factors at the SOA level that are determined to drive business success.
He also adds that it is important to keep metrics being employed down to a manageable number. “We’re not talking about hundreds of web services, enterprises can only actively manage 40 or 50 metrics effectively,” he says.

David Linthicum agrees that the important thing to do is to make sure that the business side has plenty of input into the process, as the business need should be the overwhelming priority. He emphasises that any SOA project should have a powerful business sponsor and requires the co-operation of stakeholders from all levels of the business.
To sum up then: a successful SOA project requires close co-operation between the business side and IT side of the company, with the business team taking the lead. It needs a corporate executive sponsor as well as a clear methodology, defined from the outset, which keeps the number of metrics to a minimum.