Getting Tough On ROI

Companies are taking a hard look at returns on IT investments, using complex valuation models linked to business goals.

By Eileen Colkin, InformationWeek
Oct 21, 2002 (12:00 AM)
URL: http://www.informationweek.com/story/IWK20021017S0013

Tough competition and even tighter budgets mean that IT projects must go through a rigorous ROI wringer. And that wringer is getting tougher all the time. Forget on time and on budget, and don't even think about using a vendor's ROI tool. The smartest companies are measuring a complex mix of business objectives, costs, and risks, and holding managers accountable for results that maximize returns.

"It used to be the 'ta-da' strategy," says John Howell, VP and program director of Internet solutions for Citibank Global Securities Services. "We'd put the project together and throw it out there and say, 'Ta-da! It must be successful.' We didn't look to maximize ROI, we looked to measure it."

John Howell -- Photo by Rachelle Mozman


Citibank Global Securities Services has moved beyond the easy approach to ROI with a methodology that looks to maximize returns, Howell says.

The new frontier is "more acuity in computing ROI," says Howard Rubin, a principal at Meta Group. Companies categorize IT initiatives by specific goals, such as raising the stock price, increasing market share, or lowering operating costs, then use historical and other data to quantify what returns can be expected. "IT departments need to look at the big picture," says Calvin Braunstein, president and executive research director at advisory firm Robert Frances Group. They're also tightening the links between IT investment and its impact on a company's sales and profit. Spending should go up only when revenue is headed in the same direction or costs are going down. "It has to impact either the top or bottom line," Braunstein says.

Still, only a few companies are using broader definitions of ROI. About 8% of all businesses examine IT investments through these more complex valuation filters, Rubin says. And those that are doing so use a variety of methodologies.

Chris Lofgren, president and CEO of Schneider National Inc., a $2.4 billion-a-year trucking and logistics company, has embraced the move to a more complex approach to ROI. "The emergence of the ROI metrics came from a realization in the tech community that sometimes they built things that were neat and cool because they could, even though there wasn't much value," Lofgren says. "Now there's an evolution to the extent that if companies want to push capital into a technology, IT has got to compete for that capital with proven valuation."

Chris Lofgren -- Photo by Bob Stefko


Project valuation has become more of an art than a science at Schneider National, president and CEO Lofgren says.

Lofgren puts IT investments into strategic buckets. Those that will lower costs go in one bucket, revenue creators in another, and those expected to simplify business processes in a third. He then considers different factors for each category, consulting the executives and business units relevant to each set of projects. But these sorts of valuations are still more of an art than a science. "You can't take away judgment, business strategy, and insight," Lofgren says.

Citibank Global Securities has made the transition away from the "ta-da" strategy to a more comprehensive approach to assessing the potential returns on IT projects. The company, which sells stocks and bonds to institutional investors, is building an executive portal that will let it act as a central information source and value-added service provider for C-level executives at the 350 largest financial institutions in the world. Having such a small target market leaves little room for error. One lost customer for the division is equivalent to a global retailer losing a million consumers. But the potential for gains is also huge: If the portal wins favor, Citibank's market share should increase, and it will be positioned to sell other products to this elite group, Howell says.

Howell's group built a valuation model that examined every aspect of the business to determine the best strategy for developing, implementing, and selling the portal, working with people in various lines of business to ensure that each step of the project would succeed. The goal wasn't just to come up with an ROI calculation on the project, but to design a game plan for the highest degree of success.

The biggest success factors were price and client adoption, Howell says. And Citibank wanted to get it right the first time. If it charged too little initially, customers might balk later at a price hike. But charging too much could put off potential customers. "The strategy we took was to ensure we could get the most value out of the client, maximize the money, and minimize how much we left on the table," he says.

Instead of simply weighing the costs of the implementation from an IT perspective, Howell's group worked with marketing to determine what recruitment efforts would be most effective with the target customers. "Historically, would we have put that much weight on marketing? Maybe," Howell says. But only subjectively and only at senior management's urging if they thought it might be a driver in the project's success, he says.

Toward the end of the project-valuation effort, Citibank called on a startup consulting firm to verify the predicted returns. IValue looks at shareholder value to assess ROI, using the same formulas that Wall Street analysts rely on to value companies. It builds economic simulations of IT projects that quantify things such as the demands they will place on a company's IT systems and architecture, other costs, and soft measures of success such as customer loyalty and adoption rates. Those are put into a cause-and-effect formula that traces the project's impact all the way out to its effect on the company's stock price.

The simulation can be tested by changing the cost components or demand parameters to determine what aspects will influence the project's economic performance, says Chris Gardner, who co-founded 6-month-old iValue. "Then we can make better decisions, guide development efforts, and establish priorities," he says.

IValue's simulations agreed with most of the results from Citibank's in-house valuation model. "We had to change some things in regards to how we were going to amortize the software costs," Howell says. Citibank plans to use the valuation model to evaluate future IT projects, and it hopes to cut the time needed to analyze a project to about seven weeks from the four months Howell's team spent on the portal. IValue will start selling its applications next month for businesses to create their own valuation models.

There aren't always hard numbers with which to quantify a lot of the intangible outcomes that companies look for when they implement an IT initiative, so a speculative model based on historic data is the answer, Meta's Rubin says. For instance, retention rates are a good measure of customer loyalty. A company looking to purchase loyalty-enhancing tools should build a model that correlates percentage change in customer loyalty with business goals. The models for developing these calculations need to be in place for all projects, Rubin says, so there's consistency in the decision-making process across the company.

Schlumberger Ltd., a services provider to the oil industry and an IT consultancy that spends about $500 million a year on information technology, has a new ROI-assessment process that considers an expanded set of variables. Instead of considering one estimate for the completion time of a project, the new methodology has three: a worst-case scenario, a likely case, and a best case, says Jane Walton, an IT portfolio manager.

One recent project was expected to take six months to complete and cost $6 million using Schlumberger's traditional ROI methodologies. It ended up running 20 months and $9 million. When the new ROI model was applied after the fact to the same project, it accurately predicted a worst-case scenario of $9 million. "If we had determined the range of possible outcomes and assessed the variability and risk associated with the project, our ROI analysis would have come out completely different," Walton says. "The business case wouldn't have been there."

Tim Buckley -- Photo by Bill Cramer


Shareholder benefit is what counts, Buckley says.

Companies with the most rigorous approaches to valuing IT projects use standards that link projects with the overall business strategy. At mutual-funds firm The Vanguard Group Inc., CIO Tim Buckley charts out where his company is in terms of the business goals at issue, then overlays that information with the direction a technology initiative is likely to take the firm and what the final benefit will be to its mutual-fund holders.

Vanguard has made low operating costs a centerpiece of its strategy, so spending gets high-level scrutiny. The executive team, usually including the CEO and CFO, meets in "sunlight" sessions where members debate a project's projected benefits. "We have to challenge each and every assumption," Buckley says. "You can easily get enamored with technology, and you think everyone is going to adopt it. That's where you have to have experience to give you insight into reality."

Schneider, too, relies heavily on past experience in its ROI assessments. A few years ago, the company launched an initiative to automate the selection of large dedicated truck fleets for customers, expecting that this would help it win more big deals. Schneider implemented a prototype for one customer but realized the project was failing because the trucking company's internal sales and technology operations people weren't communicating with each other. "The project was technically very good, but there were issues stopping it from working," Lofgren says. "So rather than damage a customer relationship, we pulled the plug."

Schneider is revisiting that project, but this time with the help of hindsight. Training for sales and tech employees is figured into the ROI equation, along with other factors such as maintaining a system after it's created. "A project will create a cost burden throughout its existence," Lofgren says. "That's a cost that has to create an annuity of benefit."

Valuation models involve managing projects to ensure that they're on time, on scope, and creating the anticipated value, as much as setting up metrics for returns beforehand, Meta's Rubin says. "Historically, if a project slipped, someone would ask for more time and budget. Now if it slips, you go back to the business case." If it's not meeting the goals, then it's canceled. Companies that follow this practice generally shed 40% of their projects, Rubin says. What's more, if projects are costly to maintain once implemented, they may be disrupting strategic goals and need to be re-evaluated.

Two years ago, when PeopleSoft Inc. CEO Craig Conway told CIO David Thompson to help cut $100 million in expenses for the company, Thompson launched an effort to justify the IT projects he had under way. He worked with the software company's business units to revisit old projects and track the returns they were generating. "People put together ROI savings [projections], but nobody follows through and says, 'We never achieved the goals' or holds people accountable," Thompson says.

PeopleSoft built a Web-based application (which it plans to start selling next month) that ties project portfolio management into other applications, such as human resources and finance. A project entered into the system gets input from across the company on what it will cost each business unit and how each will benefit. The investigation extends to issues such as time spent changing processes or creating and learning new ones and effects on employee productivity. Once launched, PeopleSoft can monitor a project in real time. Thompson can tell to the penny where his budget is going and who in the company is benefiting. Daily budget reports show how effective the projections are, Thompson says. "We don't have to wait for a project milestone to come around and get blindsided by it. We can take corrective action immediately."

PeopleSoft and iValue aren't alone in offering valuation tools. Among others available: Hubbard Decision Research borrows from management science, game theory, and economic equations to assess potential risks and returns of an IT investment. Enamics Inc. provides software and services for aligning IT and business goals. It plans next month to roll out a new version of its offerings that includes project portfolio management. ProSight Inc., a maker of project portfolio-management software, introduced an upgrade earlier this month that measures IT investments against business goals. IT advisory firm Gartner this summer began selling a consulting service that measures the business benefits of IT.

Of course, changing how projects are evaluated requires a change in executives' thinking, which isn't easy. "We run into a lot of resistance because there's a power shift involved," iValue's Gardner says. "Executives don't have the level of discretion they had in the past, where they could get approval and shape a project's direction. They're more constrained by facts that now enter the discussion."

IT departments should brace for new, increasingly tough demands for rigorous ROI testing of projects. While the up-front work required to get projects and purchases approved will increase, the success factor should increase as well, as IT dollars get spent more wisely.

— with Mary Hayes

Photo of Tim Buckley by Bill Cramer
Photo of John Howell by Rachelle Mozman
Photo of Chris Lofgren by Bob Stefk