Management of information technology investment: A framework based on a Real Options and Mean–Variance theory perspective

ARTICLE IN PRESS

Technovation 28 (2008) 122–134 www.elsevier.com/locate/technovation

Management of information technology investment: A framework based on a Real Options and Mean–Variance theory perspective Liang-Chuan Wu, Chorng-Shyong Ong Department of Information Management, National Taiwan University, No. 50, Lane 144, Sec. 4, Jilung Road, Daan Chiu, Taipei 106, Taiwan, ROC

Abstract The selection of appropriate technology projects has been one of the most significant business challenges of the last decade. Information technology projects, in particular, represent the largest capital expenditure items for most US firms, yet many projects have been unsuccessful. Because of the importance of such investments, there is an urgent need for a framework to analyze them. In this paper, Real Options analysis in conjunction with classical financial theory, namely, the Mean–Variance (MV) model, is used to provide new perspectives on project selection. We develop a quadripartite framework and subsume the risks within its dimensions. Furthermore, we map the corresponding options in each of the quadrants. The framework offers an easy, but comprehensive, way for managers to evaluate potential projects. In addition, we conduct a case study to demonstrate how practitioners can apply the framework. This paper contributes to the technology management field by defining the risk dimensions of technology investments, and providing insights based on interdisciplinary financial theories. r 2007 Elsevier Ltd. All rights reserved. Keywords: Framework; Project selection; Mean–Variance model; Real Options; Risk

1. Introduction Project selection has become an important issue in the technology management field (Linton et al., 2002; Shehabuddeen et al., 2006; Sun and Ma, 2005). The rapid development of technologies, together with their increasing complexity and variety, has made the task of technology selection very difficult (Shehabuddeen et al., 2006). Organizations are eager to learn how to use their limited resources effectively in order to gain a competitive advantage. To date, the literature on project selection has focused on Research and Development (R&D) investments (Coldrick et al., 2005). The question of how to select information technology projects, the largest capital expenditure items for most US firms (Kohli and Devaraj, 2004), has rarely been discussed. Appropriate information technology investments can help companies gain and sustain a competitive advantage Corresponding author. Tel.: +886 2 3366 1187; fax: +886 2 3366 1199.

E-mail address: [email protected] (C.-S. Ong). 0166-4972/$ - see front matter r 2007 Elsevier Ltd. All rights reserved. doi:10.1016/j.technovation.2007.05.011

(Melville et al., 2004). Although they form a subcategory of technology, information technology projects possess unique characteristics. It is significant that many information technology investments have proved unsuccessful, exceeded budget, and even harmed companies (Bingi et al., 1999; Chen, 2001; Somers and Nelson, 2003). Large information technology investments, such as Enterprise Resources Planning (ERP), have a failure rate as high as 75% (Griffith et al., 1999). The importance of such projects means that organizations must focus on ways to improve their implementation. However, very few studies have dealt with multi-dimensional information technology portfolio management, even though an easy-to-understand framework would help managers better understand the characteristics of different types of information technology projects. Furthermore, in the last two decades, most of the frameworks proposed for identifying information technology investment opportunities failed to capture the dynamic nature of such investments, and few dealt explicitly with the risks involved (Benjamin et al., 1984; McFarlan et al., 1983; Neumann, 1994b). Information

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

technology investments are very risky and difficult for managers to control (Kumar, 2004; Schwartz and ZozayaGorostiza, 2003; Sherer and Alter, 2004). They are undertaken in rapidly changing environments in conjunction with dynamic organizational factors, and are thus subject to many uncontrollable risks. Therefore, a framework that identifies and analyses such risks must be an integral part of the investment decision-making process (Neumann, 1994a). In this paper, we attempt to answer three key questions: 1. How can we better identify the benefits and risks associated with information technology investments? 2. How can theories from other disciplines be used? In particular, how can theories from the financial field contribute to information technology investment portfolio management? 3. How can we develop a comprehensive framework to help managers understand the opportunities and risks involved in managing a portfolio? Moreover, what options are embedded in different types of information technology investments? Options Theory (OT) is especially valuable for analyzing investments that involve both a high level of uncertainty and a large number of potential opportunities (Copeland and Antikarov, 2001). In this paper, we use Real Options and the Mean–Variance (MV) model to analyze information technology investments. The MV model is a classic financial theory that ranks assets in terms of returns and risks. We develop a quadripartite framework and subsume the investment risks within its dimensions. Furthermore, we map the corresponding options in each of the quadrants. The framework provides a comprehensive way for managers to implement and control information technology investment portfolios. It can be used to identify investment opportunities, understand the implications of specific investments from the options perspective, and prioritize information technology projects. This insight, combined with the use of interdisciplinary knowledge to manage projects, offers new perspectives on technology portfolio management. The remainder of this paper is organized as follows. Section 1 contains an overview of financial theories, including the MV model and OT. In Section 2, we explore two aspects of information technology projects in depth, namely, the benefits and the risks involved. In Section 3, we present a comprehensive framework to guide technology portfolio management. We also describe and compare the characteristics of information technology projects in each quadrant of the framework. In Section 4, we extend the framework by adding options to it. In Section 5, we conduct a case study to demonstrate how practitioners can apply the framework. Then, in Section 6, we present our conclusions, discuss the implications of the present study for researchers and managers, and indicate the direction of future research.

123

2. Literature review 2.1. Technology project selection and information technology projects The ranking and selection of technology projects is important; hence, a great deal has been written on the subject. Most works focus on R&D project selection or the selection of generic manufacturing technologies (Csaszar et al., 2006; Jolly, 2003; Lawson et al., 2006; Mikkola, 2001; Shehabuddeen et al., 2006). Many methods have been proposed for selecting R&D projects. Linton et al. (2002) published a review of R&D project selection methods. They divided R&D project selection into three categories. The first category is the traditional Net present value (NPV) method, which does not consider future uncertainties (Linton et al., 2002; Shehabuddeen et al., 2006). The second is the scoring method, which compares projects according to a number of criteria. The final category covers mathematical methods, such as data envelopment analysis (DEA) and the analytic hierarchy process (AHP) approach (Chen et al., 2006). Although there are many studies on R&D project selection, information technology projects have a number of unique characteristics that must be considered. In addition to being among the largest investments that organizations make, as well as the high failure rate and the potentially disastrous impact of such failures, information technology projects differ from other technology projects in that they are intertwined with the organizational process and structure; thus, they are also affected by organizational uncertainties. Given the increasingly dynamic nature of the business environment, information technology projects generate uncertainties, from both technological and organizational perspectives. Technological uncertainty results from the fact that information technology changes rapidly, hence projects may become obsolete much faster than organizations expect. A cutting edge information technology may be outdated by a competing information technology, or even a potential information technology rival that cannot be foreseen at the time of planning. Furthermore, wrong decisions can cause the failure of projects. Therefore, the selection of appropriate projects is becoming increasingly difficult. Organizational uncertainty is even more risky because it is endogenous. Types of uncertainty range from unforeseeable user resistance, the risk of commitment escalation, the costs of personnel turnover, to the maintenance costs incurred by the long implementation process. All of these dynamic factors make the selection of information technology projects difficult. Consequently, decisions regarding projects must be made with a great deal of caution because, unlike R&D projects, they involve a fundamental re-engineering process that results in major organizational changes.

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

124

2.2. Financial theory concepts We now introduce the concepts of the two financial theories adopted in this work and explain their implications for our framework. 2.2.1. The MV model In the financial field, several studies have been devoted to establish portfolio selection criteria. For example, Markowitz (1952) proposed criteria for constructing an ‘‘Efficiency Frontier’’ in the MV model. His model assumes that investors pursue the maximum expected rate of return relative to a given risk, or pursue the minimum risk relative to a given expected rate of return. Thus, one can distinguish between two sets of stocks on the Efficiency Frontier, i.e., the efficient solution on the A–B curve (see Fig. 1). The criteria of the MV model are: Stock k dominates stock m if gk 4gm and sk psm or mk Xmm

and

sk osm .

According to the criteria, the Efficiency Frontier should be quasi-concave. The MV model judges whether or not an asset is efficient, so it would appear that portfolios on the frontier are dominant and have a higher rate of return at the same risk level; or they have a lower risk with the same return. An asset is efficient if it has a higher rate of return, given the same risk level. The advantage of the MV model is that it provides a theoretical framework for identifying uncertainty, reducing risk, and maximizing value in IS portfolio management. In light of the discussion of the benefits and risks in the MV model, we incorporate the two dimensions into our framework. This aspect is addressed in Section 3. We explain why the central idea of the MV model combined with OT provides a new perspective for information technology investment portfolio management in the next sub-section.

2.2.2. Options Theory: an overview and basic concepts The application of OT in the finance field is a fairly recent development. Researchers had long hoped to find a rigorous way to price derivatives, but it was not until the 1970s that the ground-breaking Nobel Prize winning works by Black and Scholes (1973) and Merton (1973) achieved the goal. Based on Ito Calculus and the concept of dynamic portfolio hedging, the authors made a major breakthrough by deriving a differential equation that must be satisfied by the boundary conditions of the call option value. This resulted in the famous closed form Black–Scholes formula, which led to the rapid development of OT. OT is based on the concept that the option holder has the right, but not the obligation, to exercise an option (see Fig. 1); thus, the option’s payoffs are asymmetrically distributed due to the limited liability of the option. By their nature, options create an asymmetrical payoff. In essence, they shift the possible distribution toward a more favorable pattern, which allows the option holder to take advantage of potential benefits when taking bounded risks (see Fig. 2). Myers (1974) was the first to suggest that option-pricing theory could be applied to real assets and non-financial investments. As Real Options are derived from financial options, the initial phase of an investment project is implicitly equivalent to buying an option. Myers observed that discretionary investment opportunities, such as growth options, can capture a project’s real value. Because the Real Options method hedges risks, project managers can pre-define a maximum downside loss to limit their

Payoff Pattern

Exercise Price Underlying Asset Price

Sunk Cost

expected rate of return, r

B

A

Sunk Cost Underlying Asset Price

Exercise Price Volatility in return,  Fig. 1. The Efficiency Frontier.

Payoff Pattern Fig. 2. The concept of financial options.

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

losses. In addition, they can take advantage of unlimited upside potential benefits. OT offers a new and more realistic means of evaluating strategic opportunities and risks that traditional valuation methods, such as the NPV approach, do not consider. Myers (1974), Kester (1984), and Dixit (1995) suggested using option-based techniques to value the managerial flexibility implicit in investment opportunities. They stressed the importance of the irreversibility of most investment decisions, and the ongoing uncertainty about the environment in which these decisions are made. Kulatilaka and Marks (1988) also considered the strategic value of managerial flexibility and its option-like properties, while Trigeorgis (1988) used OT to deal with the features of, and the problems associated with, the valuation of projects. In addition, many useful OT valuation techniques have been used as alternative means of decision-making and valuation, as described by Copeland and Antikarov (2001). Given the theoretical background, and because MV theory and OT investigate the risk aspect of investments, we discuss applying the two theories to information technology investment portfolio management in the next section. 3. The nature of information technology investments In this section, we discuss the two dimensions of information technology projects, namely, their benefits and risks. 3.1. Risks—a forbidden part of benefits evaluation The benefits of information technology have long been the subject of extensive discussions (Balasubramanian et al., 1999; Chan, 2000; Chang and King, 2005; Pindyck, 1988; Santhanam and Hartono, 2003). Some works address the impact of information technology investments on an organization’s performance, while others use a causal model to present the critical success factors (CSF) of such investments. However, the ex-post view of the impact of information technology on organizations fails to capture the ex-ante view. Moreover, CSF does not describe how information technology interacts with a changing environment in a dynamic manner. Most studies have ignored the inherent value of managerial flexibility when assessing information technology investments, and used traditional evaluation methods, such as NPV, instead. However, the shortcomings of traditional project valuation tools are well documented. For example, NPV assumes that investments are reversible, and non-deferrable, but in the real world, information technology investments are irreversible, deferrable, and undertaken in conditions of uncertainty (Dixit and Pindyck, 1994; Huisman and Kort, 2002; Olafsson, 2003). Furthermore, NPV ignores the strategic value embedded in information technology investments (MacDougall and

125

Pike, 2003). A number of researchers have applied OT to information technology investments. Dos Santos (1991) and Kumar (1996) suggested that the theory could be applied to information technology investments to hedge project risks. Some researchers have employed specific OT formulas to guide information technology investments. Benaroch and Kauffman (1999, 2000) used the Black and Scholes (1973) option pricing formula to evaluate the value of deferring investments related to the expansion of electronic banking networks. Taudes (1998) applied the Margrabe (1978) formula to assess the growth opportunities of a software platform implementation. Subsequently, Kumar (2002) used the Margrabe formula to decide whether to defer a CASE tool project. These studies stress that the value of managerial flexibility should be included in the value of information technology investments. It is worth noting that an investment deemed infeasible under the NPV approach may be considered viable under the Real Options approach because it considers the value of managerial flexibility. As most companies make information technology investment decisions without an explicit understanding of Real Options (Copeland and Antikarov, 2001), our goal is to shed light on how to use Real Options when managing an information technology portfolio. Many applications of Real Options focus on options pricing issues (Boer, 2000; Dixit, 1995; Schwartz and Zozaya-Gorostiza, 2003). In this paper, however, we do not derive pricing models. Instead, we focus on improving managers’ understanding of the meaning of Real Options, what options are embedded in specific projects, and how they can best deal with those options under uncertainty. 3.2. A close-up of information technology risks Since information technology investments involve uncertainties that must be carefully managed, the relative importance of various risks should be addressed. Table 1 shows there are two kinds of uncertainty in the dynamic environment of information technology investment. The first, ‘‘external uncertainty’’ comes from outside the organization. Every company in the market faces external uncertainties, as opposed to internal uncertainties that occur within a company. According to Options Theory, the former affects the option value positively, while the latter reduces that value (Boer, 2000; Dixit, 1995). For example, every company faces uncertainty about demand in the marketplace, i.e., the risk that demand will be low or high. Exogenous risks, which are analogous to the volatility of financial options, create opportunities and increase the value of an option. Real Options are especially valuable when projects involve a high level of uncertainty combined with opportunities to dispel that uncertainty as new information becomes available (Copeland and Antikarov, 2001). Since decisions made in the implementation of information technology projects are contingent on unknown future states, OT is

ARTICLE IN PRESS Table 1 Information technology risks Risk items

Uncertainty to be resolved

External Revenue uncertainties Technology

Internal Cost uncertainties Time Specification changes Usage demand Support

Uncertainty about being over-budget Uncertainty about development time Risk of design change Uncertainty about future usage of the systems Support of senior management

suitable for redefining decision-making behavior via strategic business thinking. Although some studies recognize the use of these options in generating additional value in terms of managerial flexibility (Benaroch and Kauffman, 1999; Kumar, 2002; Sumner, 2000), there is a dearth of research into the application of Real Options to information technology investments. In the next section, we propose a framework for evaluating information technology portfolio investments in the terms of their potential benefits and specific uncertainties. 4. A framework for the analysis of an information technology investment portfolio In this section, we present a comprehensive framework to help managers better understand how to apply OT, and how to identify the opportunities and risks inherent in information technology investments. Based on the discussion in Sections 2 and 3, we combine the MV model and OT, and discuss the benefits and risks of information technology investments. 4.1. An MV perspective of information technology portfolio management In the framework’s matrix (Fig. 3), the horizontal axis represents the degree of uncertainty that an information technology project involves, i.e., the risks; while the vertical axis represents the potential benefits that could accrue from such projects. In the 2  2 matrix, the projects are mapped into four generic parts. The top right-hand quadrant in the figure represents a ‘‘Risky’’ investment that has high potential benefits and a high degree of uncertainty. This type of information

Monopoly

Risky

Moderate

Escalation

Low

The revenue stream generated is uncertain Uncertainty about acquiring needed technologies Market A revolution in which a whole market extinction disappears, such as when typewriters became obsolete due to the invention of computers Partner goes The risk of a project being halted midway bankrupt Competition A single product, sometimes known as a ‘‘killer application’’, dominates the market, squeezing out all competitors

PROFITABLITY

Uncertainty factors

High

L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

126

Static

Low

UNCERTAINTY RISK OF THE INVESTMENT

High

Dynamic

Fig. 3. The matrix of the proposed framework.

technology is usually a strategic competency-driven project, such as an ERP system. A ‘‘Monopoly’’ investment has high potential benefits and a low level of uncertainty. If a project falls in the top left-hand quadrant, it has low risks and high potential benefits. For example, a project that has first-mover advantages is a typical Monopoly investment. This type of information technology gives companies a unique competitive advantage in the market because there are no threats from a competitor, at least for a period of time. A classic example is the online reservations system first developed by American Airlines. A ‘‘Moderate’’ project has low potential benefits and low uncertainty, and falls in the bottom left-hand quadrant. A typical example is a system used for daily processing of structured data. Such investments are necessary for running the day-to-day operations of a company. The bottom right-hand quadrant indicates projects that undergo an ‘‘escalation of commitment’’. These projects have low potential and a high degree of uncertainty, and are usually unsuccessful. In fact, many information technology projects that prove to be difficult, lengthy, and over budget should be terminated before completion. Instead, they undergo budget escalation, which negates any overall advantage to an enterprise, and may even threaten a company’s survival. To reduce losses, a firm should exercise the abandon option, i.e., stop further investment in unsuccessful projects. In the next stage, we utilize the MV model to choose projects on the Efficiency Frontier, and remove those that are not efficient. A project is more efficient if it is more profitable than others with the same risk, or yields the same benefits with less risk. On the quasi-curve, information technology investment projects are subjective at points A and B (see Fig. 4); projects that fall on the frontier are the most efficient. Therefore, because resources are limited, organizations should choose technology projects on the Efficiency Frontier. The choice of these candidate portfolios has no absolute right or wrong answer, since the decision depends on the manager’s risk preference. In practice, both risk-averse and risk-seeking managers can set their goals according to their attitudes to risk. According to the MV criteria, if projects have the same risk level, managers first allocate resources to the portfolio

ARTICLE IN PRESS 127

Risky

Monopoly

PROFITABLITY

High

High Profit

L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

B

Escalation of Commitment

Moderate

No profit

Low

A

Low Static

High

UNCERTAINTY RISK OF THE INVESTMENT

Dynamic

Fig. 4. The proposed framework—an MV perspective.

Risky

High

Monopoly

B PROFITABLITY

C Moderate

Escalation of Commitment A

Low

D

E

Low Static

UNCERT AINTY RISK OF THE INVESTMENT DECISION

High Dynamic

Fig. 5. The framework of information technology portfolio management.

that will yield higher returns. A company can thus manage technology investment risks within the degree of uncertainty they can tolerate. Many large-scale technology projects, such as ERP, fail because of poor risk control (Markus et al., 2000; Sircar et al., 2000). In the first step, the MV model provides a risk-hedging perspective that helps managers allocate their portfolios. 4.2. The proposed framework combined with MV and Real Options perspectives To ensure that the framework is practical and easy to use, we map the types of options in our matrix. The options are: the growth option, the option to abandon, the option

to switch to an input mix or an output mix, the option to alter operating scales, the option to defer, and the compound option (Taudes, 1998). In the decision-making process, managers must carefully reconfigure different options when evaluating an information technology portfolio. We now distinguish the options embedded in each of the four quadrants (see Fig. 5). By determining where a project lies in the framework, managers can identify the options available to them. Projects in Portfolio ‘‘A’’ are characterized by low benefits and low risks, and are usually found in a highly competitive market. As competition reduces managerial flexibility (Dixit, 1995), the more intense the competition, the lower the value of options each company enjoys.

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

128

High Profit

The information technology systems in Portfolio ‘‘A’’ are often built to meet daily operational needs and provide managers with few options. The technology projects in Portfolio ‘‘B’’ are strategydriven. They have a high potential value and a high failure rate. If they are successful, they yield a distinct competitive advantage. For example, R&D investments are characterized by high costs and high potential value, but they also have a high failure rate. To hedge the uncertainty inherent in such projects, managers can use the ‘‘learning option’’, which calls for staged investments, i.e., a series of outlays that create the option for the next stage of the project. Each stage can be viewed as an option on the value of subsequent stages and can open up future growth opportunities. This occurs when a pilot investment is followed by further investments after uncertainties have been resolved. When large technology projects are implemented, uncertainty may be resolved by the creation of preliminary modules or prototypes, or by the development of an infrastructure that enhances future competency. This is a limited-commitment investment with an uncertain payoff that conveys the right, but not the obligation, to make further investments should the payoff look attractive. In Portfolio ‘‘C’’, a technology project is protected, and the company is free to charge an excessive ‘‘economic rent’’ that stifles competition from other companies. This occurs when a new product development (NPD) creates a new market or legal protections allow a company this competitive advantage. In this context, the corresponding option is the option to expand. The firm can expand the scale of production when market conditions are favorable.

Portfolio ‘‘D’’ projects are not as efficient as those in Portfolio ‘‘B’’ because they have lower expected benefits, but the same level of risk. However, according to OT, opportunities can be judged by their uncertainties, which indicate zones of potential value. Such uncertainties imply the possibility of good outcomes, so managers are not advised to reject projects in Portfolio ‘‘D’’ immediately. Instead, they can wait and see how the situation unfolds; thus, they hold two options in this context (Fig. 6). The first is the ‘‘defer option’’, whereby managers can delay an investment decision. They can wait to see if the environment justifies investing in the information technology. The second option signifies a switch in the input or output mix of the project. For example, in a flexible manufacturing system, the product can be adjusted if prices or demand change. The difference between Portfolios ‘‘D’’ and ‘‘E’’ is that projects in ‘‘E’’ are precarious because projected returns do not meet the cost of implementing the project. Therefore, such portfolios are characterized by high risks but low potential, which correspond to the option to abandon. This implies that if market conditions deteriorate dramatically, management can abandon current operations permanently and realize the resale value of capital equipment and other assets in the secondhand market. Abandon options include decisions to liquidate assets, exit the market, or halt investments midway. The option to abandon allows a company to predefine a maximum downsize loss based on the ‘‘stop-loss’’ concept if the environment changes radically. The conditions in Portfolio ‘‘E’’ have other effects. Based on OT, too much uncertainty raises the threshold at which a company will enter a market (Dixit,

Risky

High

Monopoly

PROFITABLITY

• Expand Option

• Learning Option

Moderate

Escalation of commitment

No profit

Low

• Option value decreases according

• Defer Option

to the degree of market

• Switch Option

competence.

• Abandon Option

Static

Low

UNCERTAINTY RISK OFTHE INVESTMENT DECISION Fig. 6. Options embedded in the framework.

Dynamic High

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

1995). For a company contemplating such an uncertain project, it would obviously be unwise to undertake the project from the perspective of ‘‘escalation of commitment’’ and the high cost threshold of entering into a quest that may not yield future benefits. We have explained how, within the dimensions of profitability and risk, the proposed framework divides technology projects into four types and maps the corresponding options. For example, the switch option and the defer option are available when implementing a project that falls in Portfolio ‘‘D’’. Since the portfolio describes a situation with vague uncertainties and benefits, a company can defer an investment and wait to see how the business environment unfolds. Though uncertain, the potential benefits may influence the company to adopt the Portfolio ‘‘D’’ project; that is, the company can defer the investment to see how the situation unfolds. Instead of the using the NPV ‘‘Invest if V4K’’ rule, we can use the following OT rule: ‘‘Invest if V4V*’’, where V* is the critical value threshold for investment. When a project involves a high degree of uncertainty, the ‘‘Hysteresis Effect’’ comes into play, so the company will delay implementing the project (Dixit, 1992). Compared with Portfolio ‘‘B’’, the uncertainty makes the wait options viable when they are viewed as opportunities. Compared with Portfolio ‘‘E’’, the potential benefits of ‘‘D’’ projects should not be rejected

129

immediately. A full list of portfolios and their corresponding options is given in Table 2. 5. Case study In this section, we describe a case study to demonstrate how a medium-sized technology equipment provider in Taiwan, hereafter referred as Company M, applied our framework’s decision-making process in the management of its project investment portfolio. Having identified five possible information technology projects, the company faced the problem of selecting the most suitable project. 5.1. The decision-making process applied by the company using the proposed framework Established in 1989, Company M focuses on the design and manufacture of electronic products, and has factories in Taiwan and Mainland China. The company offers a diverse range of products and services to clients worldwide. These include OEM projects on printed circuit boards (PCB) with multi-national technology companies, as well as PC peripherals, such as hubs, PCI host cards, card bus adapters, card readers, mobile storage devices, and media players. The company employs over 550 people, and its gross sales in 2006 were over US$24 million.

Table 2 Information technology investments and corresponding options Quadrant

Characteristics

Description

Corresponding options

Portfolio ‘‘A’’

Low potential/low risk

Intense competition reduces option value. The more intense the competition, the lower the options value for the company due to sharing the market with competitors. Systems like daily accounting systems fall into this category, as they perform a standard task similar to many other off-the-shelf products.

Reduction in option value

Portfolio ‘‘B’’

High potential/high risk

Unknown environments, such as uncertainties about the technology and the payoff, make projects very risky. This occurs when information technology investment opens up future growth opportunities and can be used as a weapon against future competition. This type of investment is usually characterized by high costs and a high failure rate, as shown by ERP investments.

Learning option (opens up growth in compound stages)

Portfolio ‘‘C’’

High potential/low risk

The company enjoys excess economic rent in the form of a monopoly. This usually occurs when an advance in technology receives a patent or other legal protection that prevents competition. The firm can expand the scale of investment in the information technology project because of market conditions are more favorable than initially expected.

Expand option

Portfolio ‘‘D’’

Both potential and risks uncertain

The company should postpone further investment until better conditions arise. An interminable situation in which the hysteresis (lag) effect occurs because of the uncertainty about opportunities that arises when investing in information technology projects. The degree of uncertainty is dependent on the likelihood of a profit. This usually occurs when critical information technology success factors are extremely uncertain (e.g., waiting until a brand-new information technology can be developed).

Switch option/defer option

Portfolio ‘‘E’’

Low potential/high risk

Significantly lower potential and higher risks make such information technology projects too expensive.

Abandon option

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

In 1998, a new business unit was set up to meet the increasing demand for touchscreen solutions. Various industrial-grade panel PCs, all-in-one POS terminals, and photo kiosks have been developed for factory automation applications. Company M has also worked on numerous ODM/OEM projects with system integrators, such as Texas Instruments and Toshiba. When it was established, Company M developed its own work procedures and several in-house information systems. However, over time, those procedures and systems became outdated and inefficient. For example, the purchasing procedure was very time consuming and involved a great deal of unnecessary paper work. Some decisions that needed approval by senior managers were often delayed when they were on business trips. Such inefficiency overloaded the employees and led to criticisms like: ‘‘It takes too long to get managers’ approval for decisions. An online approval function is really needed.’’ Similar complaints arose in the accounting and bidding process. The single-PC accounting system, which ran in isolation from other systems, such as the purchasing system, could not provide updated real-time data. All related data had to be gathered by the accounting staff manually. Such problems meant that the company could not respond rapidly to changing market conditions or customers’ needs. Because of its rapidly expanding business, Company M decided to invest in new equipment and technologies, and refine the production process to meet shorter lead-times, speed up the processing of customers’ orders, and thereby generate higher profits. The objective was to build on the close supply chain relationships it had developed and improve efficiency by integrating all company functions in real time. After an in-depth study of the company’s needs, five candidate projects were short-listed by senior management. They were the ERP project, the IBM Notes project, the Supply Chain Management (SCM) project, a project to develop a portal site, and a project that involved Company M updating the existing systems by itself. Although the company received the equivalent of US$30 000 in government subsidies to complement its own budget of US$700 000, the amount was clearly not sufficient to support all of the candidate projects. Some could be implemented almost immediately, while others should be rejected or at least delayed. In addition, due to the high failure rate of information technology projects in this industry, Company M was acutely aware of the importance of strictly controlling the risk of going over deadline or over budget.

company streamline its business processes by creating an enterprise-wide transaction structure that integrated the key functions of different departments. Therefore, the management considered the ERP project to be the most promising because it had the most potential to develop and sustain a competitive advantage in the market. In contrast, the management decided that committing more resources to update the existing system would generate few benefits. In fact, it would incur huge future maintenance costs because of the need to standardize inconsistent data formats and streamline inefficient work practices. It was decided that the SCM project had the potential to help the company improve customer satisfaction and its relationship with its suppliers. These factors would improve the company’s competitiveness and thereby increase profits. The Portal Site project was deemed the least risky of the five candidates because implementing it would involve little ambiguity or uncertainty about users’ requirements. However, since almost all companies in the industry have similar systems, the return on implementing this project would be limited. The Notes project, which provides a coordinated work environment, was also considered by the management as a way of improving work efficiency. Moreover, the estimated return was higher than the cost of fixing the outdated system. The return of the five projects was estimated based on the combined direct savings and the indirect annual savings due to a shorter payback period, savings on labor costs, and improvements in productivity. The rankings of the five projects based on the estimated return data are shown in Fig. 7. The other parameter of MV analysis, i.e., the risks posed by the five projects, was also considered. A number of methods can be used to calculate such risks. One approach determines risk based on public market data, such as listed companies involved in similar projects (Campbell, 2002). Management expertise is another widely used method (Kumar, 2002; Taudes et al., 2000). Benaroch (2002) proposed a method that breaks risk down into risk factors, which are measured separately using the above-mentioned

S

5.2. Stage 1: MV analysis The first step for Company M was to prioritize the five projects and conduct an MV return and risk analysis of each candidate. The process of estimating the return is similar to traditional, widely used NPV calculations (Kumar, 2004). Among the five projects, the ERP project could help the

E: ERP N: Notes P: PortalSite S: SCM O: Old System

E Return

130

N P O

Risk Fig. 7. Application of the MV model to Company M.

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

methods. Moreover, Bardhan et al. (2004) developed a method that estimates a project’s risk based on different project scenarios, without resorting to historical data or the need to make ad hoc project-specific assumptions. Company M ranked the risks of the five projects based on their expertise and knowledge of the industry. Among the five projects, the ERP project was ranked as most risky because of possible cost/schedule overruns during the lengthy and costly implementation process. The IBM Notes project offered several functions that were similar to those of ERP and met Company M’s requirement for an online coordinating function that would speed up work processes. However, although this choice was less expensive and less risky than the substantial ERP investment, it was decided that it could not meet all of the company’s needs. Some critical functions, such as cross-function data, and real-time workflow were not available to cope with the anticipated rapid growth in business worldwide. The management also considered the risk involved in fixing and updating the existing system. The project would mean switching from a PC-based environment to a Webbased one, which would require a tremendous amount of work and strain the company’s limited resources. After a thorough examination, the management decided that the project was not a viable option. It was also recognized that updating the existing system involved more risks than adopting ERP. The management also evaluated the risks of the SCM project and the new portal website project. Specifically, the goal of the SCM project is to strengthen a company’s relationships with its suppliers and customers. Because the complexity that the project would involve consulting Company M’s suppliers, the project was deemed too risky. The last project involved building a new portal website to replace the old company website. Since the user requirements and specification for the project were well defined, Company M had sufficient experience and resources to implement it. It was assessed as having the lowest risk among the five projects. The risk ranking of each project is also shown in Fig. 7. Because projects on the Efficiency Frontier are superior in terms of return and risks, the ERP project and the Portal Site project appeared to be best choices for Company M. The other three projects fell below the Efficiency Frontier, and were therefore considered weak in terms of return and risks. For example, it was determined that the project to update the existing system would involve higher risks and yield a lower return than the Notes project. MV analysis suggested rejecting the SCM and Notes projects, as well as the project to update the existing system. 5.3. Stage 2: options analysis Options analysis was conducted in the second stage to reinforce the MV perspective, which does not consider uncertainty. In this stage, the management ranked the

131

projects by comparing the options values embedded in them. The ERP project was deemed the most promising in terms of options values. The first option embedded in the ERP project related to learning options, which would allow Company M to avoid the enormous risks of a one-time implementation. ERP implementation is an ongoing business re-engineering process, rather than a one-off installation of a software program. The requirements evolve over time, and many ERP adopters rely on vendors for extensive technical assistance, emergency maintenance, updates, and special modifications. Through such learning, additional knowledge about the uncertainties can be gained through the initial implementation of the ERP project. Following the basic ERP implementation, follow-up investments may be made to enhance the value of ERP. Additional modules can be added depending on how the environment evolves. The second option embedded in the ERP project for Company M was the option to abandon. The company could abandon the project if the implementation evolved unfavorably. The value of this kind of flexibility is that the management can predefine the maximum sustainable losses and prevent over-commitment of resources. ERP projects that go over time and budget can be controlled or abandoned, preventing an unlimited commitment of resources that would eventually exhaust the organization’s resources. The SCM project had the option to defer. The project ranked below the approval threshold in the MV stage and was therefore not suitable for Company M to undertake right away. However, it was deemed as having the potential to improve Company M’s competitiveness because only a few of the company’s rivals had implemented SCM. Consequently, the SCM project was classified as a monopoly project, as shown in Fig. 7. The management decided to defer the project, instead of rejecting it immediately, which would be the case under the NPV rule. The options analysis suggested that the other three projects were not appropriate. The Notes project provided few options because its functions could not meet the company’s needs. The Notes project was inferior in both the MV stage and the Options stage, and was therefore rejected. The options value of the project to update the existing system was nearly zero because the project had little potential to yield future benefits. Finally, the options value of the Portal Site project was limited because of its non-strategic role and well-defined user requirements. Most rivals of Company M had similar systems. As noted in Section 4.2, competition reduces managerial flexibility and there was little option value embedded in the project. The two-stage analysis, which combined the MV analysis and the Options perspective, suggested that Company M should implement the ERP project. Although embedded with little options value, the Portal Site project was suggested by the MV analysis. Eventually, the management

ARTICLE IN PRESS 132

L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

of Company M approved the ERP project, the first choice suggested by both stages. In addition, it was decided that Portal Site project should be implemented, since the budget was sufficient and the goals of the two projects did not conflict. The SCM project was deferred and the other two projects were rejected. The feedback from the company is presented in the Appendix. 6. Discussion and conclusions Because the value of Real Options is equal to traditional NPV plus the value of future opportunities (Trigeorgis, 1996), options do not add value to every type of information technology investment. For some projects, Real Options add little value in terms of future opportunities; therefore, the options value can be measured by traditional NPV directly. Uncertainty is the key to determine which investments the Real Options approach can be applied to. It does not play a key role in every information technology investment decision. This is particularly true of small-scale applications. Simple, welldefined applications, such as daily accounting systems and office automation systems (OAS) are designed to replace workers who perform repetitive tasks, e.g., payroll clerks. These applications are well suited to NPV analysis because their costs and benefits can be determined relatively easily and users’ requirements are clear (Martinsons et al., 1999; Stefanou, 2004); therefore, they offer few options. Options are also useful for evaluating information technology projects that take a long time to implement. This is another source of uncertainty because of the dynamic nature of the business environment. When a technology evolves over several years, the potential revolution in standards can produce an entirely new paradigm, leading to an unbridgeable gap between the old and the new. Doubling the implementation time more than doubles the uncertainty. In this paper, we have considered technology investments from a different perspective by using insights gained from financial theory, namely Real Options, to enhance the understanding of technology investment portfolio management. This has important implications for researchers and practitioners. For researchers, in contrast to a large body of the literature, our paper does not focus on pricing issues. On the normative side, we first provide a practical way to realize the nature of the OT and MV. Second, we contribute to the literature by incorporating a risk dimension parameter in the framework. We provide insights on how to incorporate financial theories in order to broaden and deepen interdisciplinary discussion in the technology management field. For practitioners, our study provides guidelines for managing technology investment projects. Managers face the difficulty that most technology investments are inherently risky, especially in a rapidly changing technological and deregulated economy. Our framework provides a simple and comprehensive investment management tool.

Managers can easily understand how to use the proposed framework to assess their technology portfolio requirements. More importantly, as our framework maps different kinds of options, so managers can easily identify appropriate options and make appropriate decisions about different kinds of technology investments. To the best of our knowledge, this work is the first to combine MV theory and OT in the technology management field. Since the application of OT to the technology field is still in the early stages, future research could include studies of different technology project selection problems, e.g., in non-information technology projects. By identifying the unique features of different technology projects, researchers can study the applicability of the proposed framework to those projects, and it is our hope that the discussions, issues, and ideas set forth in the paper will motivate further studies. Acknowledgments The authors wish to express their appreciation to Dr. Jonathan Linton and the anonymous reviewers for their constructive suggestions, which have helped us improve the manuscript. Appendix. Feedback from the MIS Director of Company M In subsequent feedback, the Director made the following observations: (i) The framework improved communications First, he noted that the framework improves communications with the top management. ‘‘Our company is a typical medium-sized manufacturing company. Like many privately owned companies in Taiwan, the way we evaluate a project begins from the proposal of a project by our General Manager, and we are responsible for providing suggestions. In most circumstances, the decision processes were fast, mostly based on subjective judgments. Although we were somehow aware of the return and the risk aspects of our projects, the two aspects were still an implicit concept to be put into analysis.’’ He added: ‘‘The frame work is an easyto-understand tool for choosing projects. Our CEO is not an IT specialist. It simplified the features of the different projects and it was easy to explain the ranking procedure to our top manager. We were able to present the differences of each project to the top management. (ii) Feedback on investments Second, in his opinion, the more substantial the investments, the more valuable the dynamic project review process can be. ‘‘Selecting ERP parameters is a long-term trial and error process. We are still trying the numerous parameters provided by the ERP system and it takes time to see the effects of making adjustments. We cannot make an immediate judgment about whether the ERP project is successful or not. Many

ARTICLE IN PRESS L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

companies do not admit their ERP projects have failed to meet their needs until after a period of adjustment. Therefore, the ERP project may be in a quite different place in the framework in later evaluations.’’ (iii) Challenges/unresolved issues He also highlighted certain challenges in using the framework. The first is how to quantify the issues. ‘‘We used to report the technical feasibility of projects and our suggestions to top managers. Sometimes we made our decisions based on intuition. The required quantity estimation generated additional work especially for our MIS department.’’ ‘‘What interested the top manager most were the numbers, thus, this seems to be unavoidable for us.’’ He also noted that determining the risks is another challenge. ‘‘There was no problem in determining that the portal site is the least risky project for us. However, different opinions arose on the risk ranking of the ERP project and the Notes project. Some people thought the Notes project could be more risky for us because we could abandon ERP and stay with our existing process. However, if we were to abandon Notes, our workers may lose a working process standard. We cannot do without Notes once we have adopted it.’’ He concluded that agreement on the risk ranking of the projects is important in using the framework and ‘‘Expertise and quantitative methods may help in this situation.’’ (iv) Suggestions for improvement The MIS Director stated that ‘‘power’’ could be a part of the quadrant analysis. He noted that: ‘‘An SCM project involves a battle for power between a buyer and its suppliers. Major companies have the power to ask their suppliers to apply a bar code system to meet their needs. However, it is impossible for minor buyers to do so. Therefore, the SCM project can be in different quadrants for different companies. SCM systems are now mostly used in service industries. There might be an opportunity for us to adopt an SCM, but this decision depends on our competitive status in the future.’’ ‘‘An SCM project is an opportunity, not a burden. We can decide whether to implement SCM or not even after the ERP project is finished.’’ (v) Miscellaneous Finally, he mentioned that when determining the risks of projects, there may be a tendency to weight projects that involve huge investments with more risk. Therefore, the potential bias in overweighting the risks of such projects may be worth noting if the risk assessment is based on purely subjective judgments. References Balasubramanian, P., Kulatilaka, N., Storck, J., 1999. Managing information technology investments using a real-options approach. Journal of Strategic Information Systems 9 (1), 39–62.

133

Bardhan, I., Bagchi, S., Sougstad, R., 2004. Prioritizing a portfolio of information technology investment projects. Journal of Management Information Systems 21 (2), 33–60. Benaroch, M., 2002. Managing information technology investment risk: a Real Options perspective. Journal of Management Information Systems 19 (2), 43–84. Benaroch, M., Kauffman, R.J., 1999. A case for using Real Options pricing analysis to evaluate information technology project investments. Information Systems Research 10 (1), 70–86. Benaroch, M., Kauffman, R.J., 2000. Justifying electronic banking network expansion using Real Options analysis. MIS Quarterly 24 (2), 197–225. Benjamin, R.I., Rockart, J.F., Morton, M.S.S., Wyman, J., 1984. Information technology—a strategic opportunity. Sloan Management Review 25 (3), 3–10. Bingi, P., Sharma, M.K., Godla, J.K., 1999. Critical issues affecting an ERP implementation. Information Systems Management 16 (3), 7–14. Black, F., Scholes, M., 1973. Pricing of options and corporate liabilities. Journal of Political Economy 81 (3), 637–654. Boer, F.P., 2000. Valuation of technology using ‘‘Real Options’’. Research–Technology Management 43 (4), 26–30. Campbell, J.A., 2002. Real Options analysis of the timing of IS investment decisions. Information and Management 39 (5), 337–344. Chan, Y.E., 2000. IT value: the great divide between qualitative and quantitative and individual and organizational measures. Journal of Management Information Systems 16 (4), 225–261. Chang, J.C.J., King, W.R., 2005. Measuring the performance of information systems: a functional scorecard. Journal of Management Information Systems 22 (1), 85–115. Chen, H.H., H.I. Lee, A., Tong, Y., 2006. Analysis of new product mix selection at TFT-LCD technological conglomerate network under uncertainty. Technovation 26 (11), 1210–1221. Chen, I.J., 2001. Planning for ERP systems: analysis and future trend. Business Process Management Journal 7 (5), 374–386. Coldrick, S., Longhurst, P., Ivey, P., Hannis, J., 2005. An R&D options selection model for investment decisions. Technovation 25 (3), 185–193. Copeland, T., Antikarov, V., 2001. Real Options—A Practitioner’s Guide. Texere LLC, New York. Csaszar, F., Nussbaum, M., Sepulveda, M., 2006. Strategic and cognitive criteria for the selection of startups, pp. 151–161. Dixit, A., 1992. Investment and hysteresis. Journal of Economic Perspectives 6 (1), 107–132. Dixit, A., 1995. Irreversible investment with uncertainty and scale economies. Journal of Economic Dynamics and Control 19 (1–2), 327–350. Dixit, A., Pindyck, R.S., 1994. Investment Under Uncertainty. Princeton University Press, New Jersey. Dos Santos, B.L., 1991. Justifying investment in new information technologies. Journal of Management Information Systems 7 (4), 71–89. Griffith, T.L., Zammuto, R.F., Aiman-Smith, L., 1999. Why new technologies fail? Industrial Management 41 (3), 29–34. Huisman, K.J.M., Kort, P.M., 2002. Strategic technology investment under uncertainty. Or Spectrum 24 (1), 79–98. Jolly, D., 2003. The issue of weightings in technology portfolio management. Technovation 23 (5), 383–391. Kester, W.C., 1984. Todays options for tomorrows growth. Harvard Business Review 62 (2), 153–160. Kohli, R., Devaraj, S., 2004. Realizing the business value of information technology investments: an organizational process, pp. 53–68. Kulatilaka, N., Marks, S.G., 1988. The strategic value of flexibility— reducing the ability to compromise. American Economic Review 78 (3), 574–580. Kumar, R.L., 1996. A note on project risk and option values of investments in information technologies. Journal of Management Information Systems 13 (1), 187–193.

ARTICLE IN PRESS 134

L.-C. Wu, C.-S. Ong / Technovation 28 (2008) 122–134

Kumar, R.L., 2002. Managing risks in IT projects: an options perspective. Information and Management 40 (1), 63–74. Kumar, R.L., 2004. A framework for assessing the business value of information technology infrastructures. Journal of Management Information Systems 21 (2), 11–32. Lawson, C.P., Longhurst, P.J., Ivey, P.C., 2006. The application of a new research and development project selection model in SMEs. Technovation 26 (2), 242–250. Linton, J.D., Walsh, S.T., Morabito, J., 2002. Analysis, ranking and selection of R&D projects in a portfolio. R and D Management 32 (2), 139–148. MacDougall, S.L., Pike, R.H., 2003. Consider your options: changes to strategic value during implementation of advanced manufacturing technology. Omega—International Journal of Management Science 31 (1), 1–15. Margrabe, W., 1978. Value of an option to exchange one asset for another. Journal of Finance 33 (1), 177–186. Markowitz, H., 1952. Portfolio selection. Journal of Finance 7, 77–91. Markus, M.L., Axline, S., Petrie, D., Tanis, C., 2000. Learning from adopters’ experiences with ERP: problems encountered and success achieved. Journal of Information Technology 15 (4), 245–265. Martinsons, M., Davison, R., Tse, D., 1999. The balanced scorecard: a foundation for the strategic management of information systems. Decision Support Systems 25 (1), 71–88. McFarlan, F.W., McKenney, J.L., Pyburn, P., 1983. The information archipelago—plotting a course. Harvard Business Review 61 (1), 145–156. Melville, N., Kraemer, K., Gurbaxani, V., 2004. Information technology and organizational performance: an integrative model of it business value, pp. 283–322. Merton, R.C., 1973. Theory of rational option pricing. Bell Journal of Economics 4 (1), 141–183. Mikkola, J.H., 2001. Portfolio Management of R&D Projects: Implications for Innovation Management. Elsevier Science, Amsterdam, pp. 423–435. Myers, S.C., 1974. Interactions of corporate finance and investment decisions—implications for capital budgeting. Journal of Finance 29 (1), 1–25. Neumann, S., 1994a. Strategic Information Systems: Competition Through Information Technologies. McMillan College Publishing, New York. Neumann, S., 1994b. Strategic Information Systems: Competition Through Information Technologies. McMillan College Publishing, New York. Olafsson, S., 2003. Making decisions under uncertainty—implications investments for high technology investments. BT Technology Journal 21 (2), 170–183. Pindyck, R.S., 1988. Irreversible investment, capacity choice, and the value of the firm. American Economic Review 78 (5), 969–985. Santhanam, R., Hartono, E., 2003. Issues in linking information technology capability to firm performance. MIS Quarterly 27 (1), 125–153. Schwartz, E.S., Zozaya-Gorostiza, C., 2003. Investment under uncertainty in information technology: acquisition and development projects. Management Science 49 (1), 57–70. Shehabuddeen, N., Probert, D., Phaal, R., 2006. From theory to practice: challenges in operationalising a technology selection framework. Technovation 26 (3), 324–335. Sherer, S.A., Alter, S., 2004. Information systems risks and risk factors: are they mostly about information systems? Communications of AIS 14 (2), 29–62.

Sircar, S., Turnbow, J.L., Bordoloi, B., 2000. A framework for assessing the relationship between information technolgy investments and firm performance. Journal of Management Information Systems 16 (4), 69–98. Somers, T.M., Nelson, K.G., 2003. The impact of strategy and integration mechanisms on enterprise system value: empirical evidence from manufacturing firms. European Journal of Operational Research 146 (2), 315–338. Stefanou, C.J., 2004. A framework for the ex-ante evaluation of ERP software. European Journal of Information Systems 10 (4), 204–215. Sumner, M., 2000. Risk factors in enterprise-wide/ERP projects. Journal of Information Technology 15 (4), 317–327. Sun, H., Ma, T., 2005. A packing-multiple-boxes model for R&D project selection and scheduling. Technovation 25 (11), 1355–1361. Taudes, A., 1998. Software growth options. Journal of Management Information Systems 15 (1), 165–185. Taudes, A., Feurstein, M., Mild, A., 2000. Options analysis of software platform decisions: a case study. MIS Quarterly 24 (2), 227–243. Trigeorgis, L., 1988. A conceptual options framework for capital budgeting. Advances in Futures and Options Research (3), 145–167. Trigeorgis, L., 1996. Evaluating leases with complex operating options. European Journal of Operational Research 91 (2), 315–329.

Liang-Chuan Wu is a Doctoral student in the Department of Information Management, College of Management, National Taiwan University, Taiwan. He also received his Master’s degree from NTU. His research interests include ERP Management, E-business, and Knowledge Management. He has published papers in Decision Support Systems, Journal of Portfolio Management, and other journals.

Chorng-Shyong Ong is a professor of Information Management at National Taiwan University, Taiwan. He holds a master’s degree in Management Science and Policy Studies at TSUKUBA University in Japan. He received his Ph.D. in Business Administration from NTU. His research interests include IS Service Quality, Web-Based Services, Electronic Commerce and Strategic Management of e-Business. He has published papers in Information & Management, Computers in Human Behavior, Journal of the Operational Research Society, Expert Systems with Applications, Applied Mathematics and Computation, Pattern Recognition Letters, Journal of Information Management, Journal of Quality, and other journals.