Opinion: Assume nothing. Audit instead
- 10 July, 2003 22:00
The IT manager who assumes that a technology investment will meet or exceed an economic or return on investment forecast reminds me of the anecdote about the economist trapped in a burning building. "No problem," he says. "I'll assume a fire hose." While the dismal science of economics relies on assumptions to build theories to explain economic phenomena, IT managers are under no such obligation. In fact, the practice might be dangerous.
One of the most damaging assumptions an IT manager can make is that the cost savings or revenue increases forecast in an ROI exercise will materialize on schedule, or at all. Yet it happens all the time.
How else to reconcile the fact that most organizations don't bother to confirm that all the economic returns depicted in a forecast actually materialize? Too often, the effort invested in constructing such a forecast as a decision-making tool becomes digital dust on someone's hard drive after the project has gone live. The thinking goes: The forecast is irrelevant because the project is done -- let's move on.
What's likely lurking behind the absence of postinvestment auditing at most companies is fear, uncertainty and doubt that the real-world results will fall below forecast. No one enjoys evaluations, especially when the outcomes are less than expected, and people in a position of accountability -- the project champions and those who control the budget -- fear a metaphorical courtyard stoning.
There are ways to conduct a consistent, repeatable postinvestment audit, and no one need fear being shot at dawn. If you accept the lofty principle that one of employees' most powerful capabilities is the ability to learn and to turn the learning process into improvements, then you should see audits as an effective component of IT management.
The maximum value from an IT investment isn't realized from simply confirming that the returns match the forecast, but rather from improving the technology asset's performance should it turn out that the returns are lower than planned -- which is often the case with more strategic kinds of technology requiring process and organizational change. The act of investigating why some element of the deployment isn't returning the projected cost savings or increased revenue is the first step of the remediation process by which maximum value is extracted from the technology. This might be the single strongest argument for devoting the time and money to an auditing process.
Consider the example of a hospital that decided to expand its lab business. Instead of doing the lab work of hospital patients only, it took on referral business from physicians and clinics as a way to increase revenue by leveraging unused lab capacity in the off hours. This was a big-risk, big-payoff project, involving an investment in people, processes and technology -- an ideal candidate for a postinvestment forecast validation exercise.
The hospital correctly anticipated that the specialized lab management software it rolled out to support this venture might introduce some business-process incompatibilities that could drag down forecast ROI. However, it had little visibility into what those exact incompatibilities would look like and how they would affect returns. The hospital's IT team decided to conduct an audit concurrent with technology deployment. It could have waited to audit the returns until some point after deployment, but the risks were high enough that it sought to catch any operating dysfunction quickly.
The audit team discovered soon after the platform went live that the software collected far more data than it needed for referral patients. The system required lab workers to capture 20 data elements from each patient with a lab order. This demanding screening process was optimized for in-hospital patients, but for referral patients, the lab really needed only a few pieces of information such as name, age and insurance carrier.
The dysfunction wasn't catastrophic, but it clearly could have compromised the volume of referral patient traffic the lab was equipped to handle. Having discovered through an audit the operational tension between the needs of the referral patient service and the information acquisition overkill of the software, the IT organization recognized the potential drag on revenue and went to work immediately on a code fix.
Today, this reference lab is a US$7 million profit center for the hospital. In the absence of an audit capability, it would have been less profitable, because lab personnel would continue to grill patients as if they had committed a crime rather than gotten sick. w John Berry is an IT management consultant and analyst in Bend, Ore. He's currently writing a book about the measurement of intangible assets. Contact him at firstname.lastname@example.org.-- Computerworld (US)