Performance measurement of a transportation network with a downtown space reservation system: A network-DEA approach

Transportation Research Part E 47 (2011) 1140–1159

Contents lists available at ScienceDirect

Transportation Research Part E journal homepage: www.elsevier.com/locate/tre

Performance measurement of a transportation network with a downtown space reservation system: A network-DEA approach q Y. Zhao a, K. Triantis a,⇑, P. Murray-Tuite b, P. Edara c a

Grado Department of Industrial and Systems Engineering, Virginia Tech, Northern Virginia Center, 7054 Haycock Road, Falls Church, VA 22043-2311, USA Via Department of Civil and Environmental Engineering, Virginia Tech, Northern Virginia Center, 7054 Haycock Road, Falls Church, VA 22043-2311, USA c Department of Civil and Environmental Engineering, University of Missouri-Columbia, E3502 Lafferre Hall, Columbia, MO 65211, USA b

a r t i c l e

i n f o

Article history: Received 14 December 2010 Received in revised form 21 January 2011 Accepted 8 February 2011

Keywords: Transportation network Performance measurement Travel demand management Data Envelopment Analysis (DEA) Network DEA

a b s t r a c t We propose an evaluation approach for a novel travel demand management strategy known as the downtown space reservation system (DSRS). This approach takes into account three perspectives, i.e., transportation service provider’s, the user’s, and the community’s and is based on network-Data Envelopment Analysis (DEA) where the perspectives are inter-related through intermediate inputs/outputs. Two types of network-DEA models (radial and slacks-based models) are considered. An example is provided using data propagated from a microscopic traffic simulation model of the DSRS. The results show that individual node performance can drive network DEA performance and that this information can inform future designs of the DSRS. Published by Elsevier Ltd.

1. Introduction Transportation systems impact stakeholders in different ways. For example, travelers are concerned with their travel time, transportation service providers strive to meet their financial constraints, and communities focus on safety and environmental issues. Conflicting interests make the traditional economic evaluation methods such as benefit–cost analysis, inadequate to concurrently capture multiple perspectives (e.g., the provider’s, the community’s, and the user’s perspectives) that are important for the performance assessment of transportation systems. Furthermore, the consideration of multiple concurrent perspectives is important for many transportation agencies that have begun to support travel demand management (TDM) strategies, which facilitate traffic congestion mitigation, environmental protection, and energy conservation. In order to address this need, the objective of this research is the development of a transportation performance measurement approach with an application of a relatively new efficiency measurement methodology, namely network DEA. Our proposed approach expands on Färe and Grosskopf’s network DEA approach (2000) and captures the perspectives of transportation system providers, the users and the community, as well as the interrelationships among these perspectives. It also provides an overall performance (efficiency) measure for the transportation network. For validation and completeness purposes, the current research compares two different types of network DEA models, the original network DEA model proposed by Färe and Grosskopf (2000) and the slacks based network DEA model proposed by Tone and Tsutsui (2009). q This paper is based in part on work supported by the National Science Foundation, while working at the Foundation. Any opinion, finding, and conclusions and recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the National Science Foundation. ⇑ Corresponding author. Tel.: +1 703 538 8446; fax: +1 703 538 8450. E-mail address: [email protected] (K. Triantis).

1366-5545/$ - see front matter Published by Elsevier Ltd. doi:10.1016/j.tre.2011.02.008

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1141

The essence of our approach is to compare and contrast various instances (scenarios) that occur in the transportation network under the execution of a TDM strategy, namely the downtown space reservation system (DSRS). The scenarios constitute the production possibility set for our analysis. Traffic flow of a downtown area, where the DSRS is implemented, is simulated with a microscopic approach. The data that support the performance measurement analytical approach is obtained from the execution of the micro simulation (Zhao et al., 2010a). The remainder of this paper is organized as follows. Section 2 presents a brief background of the DSRS and the microscopic traffic simulation. Overview of transportation performance measures and the three-perspective conceptual performance measurement approaches are provided in Section 3. Section 4 introduces the DEA and Network DEA approaches, followed by mathematical descriptions of the original network DEA and the slacks based network DEA with inclusion of undesirable outputs in both models. In Section 5, an illustrative example is presented together with the comparison between the two different network models. Finally, conclusions and future system design and research directions are addressed in Section 6.

2. Background The transportation network evaluated in this research is characterized by the implementation of the TDM strategy, namely the downtown space reservation system (DSRS). It should be noted that this system has not been implemented. The DSRS has been developed for the purpose of congestion mitigation, for the center of a city or Central Business District (CBD). With the DSRS, travelers who want to drive in a designated downtown area have to book their time slots before making their trips. The transportation authority, who administers and supervises the DSRS, allocates time slots to travelers based on the availability of resources (i.e., road network capacity). Only those who get permission from the transportation authority can drive in the downtown area during the requested time period. The system alleviates traffic congestion by reducing excessive vehicles on the road. Details of the DSRS including implementation issues are addressed by Zhao and Triantis (2009) and Zhao et al. (2010a,b). The core of the DSRS is an optimization module that maximizes two objectives, i.e., people throughput and revenue obtained from the reservation system where both objectives are subject to the transportation network capacity. In the optimization module, the decision maker assigns weights to the two objectives reflecting their relative importance. In the DSRS, different vehicle types have different vehicle occupancy rates. The people throughput is the total number of travelers that the transportation system services. In work that followed this initial research work, a microscopic traffic simulation model was built to obtain a better understanding of the transportation system behavior, to evaluate performance of the DSRS, and to evaluate the impact of various changes (e.g., changes in travel demand, etc.). The simulation model allows one to test the DSRS before it is implemented by providing a range of effectiveness measures (such as travel time and average speed). The simulation was based on a transportation network shown in Fig. 1, and it was run under different scenarios characterized by varying travel demand levels and by changing the relative importance of the people throughput and revenue objectives in the reservation system (Zhao et al., 2010b). The simulation provides the data for the network DEA model and this will be illustrated in Section 5.

Fig. 1. Transportation network for traffic simulation.

1142

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

3. Conceptual framework 3.1. Transportation performance measurement Selecting appropriate performance measures is crucial in performance measurement. Additionally, the different stakeholder perspectives and the achievement of their expectations are fundamentally reflected by the performance measures. The relationship among the measures and stakeholder perspectives is captured by the structure of the performance measurement approach that is described in Section 3.2. In terms of defining appropriate performance measures, there is a substantial literature that provides common performance measures used by different transportation agencies. The Oregon Department of Transportation (Reiff and Gregor, 2005) compiled a list of 750 performance measures that encompasses different policy areas including mobility, accessibility and sustainability. The Texas Transportation Institute (TTI, 2005) emphasizes monitoring a family of mobility measures, such as travel delay, travel time index, and buffer index. These measures essentially are similar to the measures used for congestion measurement (Cambridge Systematics, 2005). In addition, sustainability has been advocated in order to take into account environmental issues. According to a set of working definitions of sustainability in transportation, Jeon and Amekudzi (2005) recommended a three-dimensional sustainability framework including economic development, environment preservation and social development, and provided a list of performance measures for each dimension. Litman (2008) also suggested a list of economic, social and environmental indicators for assessing sustainable transportation. To account for the measures identified in the literature, this research categorizes the measures into inputs, outputs and outcomes for the transportation system as depicted in Fig. 2, where inputs are investments, outputs are direct achievements/ services, and outcomes indicate the effects that the outputs fundamentally have for the user and the community. This mapping of performance measures to the concepts of inputs, outputs and outcomes is the foundation for developing the transportation network structure discussed in Section 3.2.

3.2. Transportation performance measurement network structure Researchers and practitioners developed various frameworks for demonstrating the underlying relationships between the performance of the transportation system and multiple stakeholder benefits and expectations. Tsolakis and Thoresen (1998)

Input

Infrastructure Cost Travel Demand

Output

Average Speed Average Delay LOS Revenue

Outcomes

Environmental Sustainability

Emissions Pollutions

Social Sustainability

Traffic Accidents Accident Fatalities Accessibility

Economic Sustainability

Travel Costs

Individual Level : Delay per Traveler Travel time Travel time index Buffer index Mobility Area Level : Total Delay Congested Travel Percent of Congested Travel Congested Roadways

Transportation System Effectiveness

Throughput

Fig. 2. Transportation input–output-outcome system.

Vehicle Miles Traveled Person Miles Traveled Vehicles Hours Traveled Person Hours Traveled

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1143

suggested a four-dimension performance framework, which accommodated a range of community values and viewpoints including economic, social, safety and environmental concerns, stressing the importance of community participation and public openness in road performance evaluation. Falcocchio (2004) presented a conceptual framework for more effective development and application of performance measures in transportation system evaluations, where the interests of customers, the community, transportation providers and professional societies are included. Sheth et al. (2007) proposed a transit performance evaluation framework in conjunction with Data Envelopment Analysis (DEA) that combines the views of the transit provider, the consumer and society. Expanding on the literature, this research adopts a three-perspective framework, i.e., the provider, user and community perspectives. These three perspectives represent important stakeholders in transportation systems. Their perspectives and interactions determine the overall performance of the transportation system. Users and community stakeholders are more likely to be outcome oriented whereas providers are output oriented. Furthermore, we assume that users are more concerned about their mobility and this is reflected with the travel time related measures. Transportation service providers are mostly interested in the system efficiency and effectiveness, which is reflected by revenue, level of service, and vehicle miles traveled. Last but not least, the community typically cares more about the environment and safety issues that are associated with the traffic. Therefore, sustainability oriented measures are more appropriate to reflect their interests. The performance network of Fig. 3 represents the underlying structure of a transportation system with respect to the different perspectives and the interrelationships among these perspectives. The network consists of five nodes. Node 0 and node 4 are dummy nodes. The major function of these nodes is to distribute inputs to and collect outputs from the intermediate nodes (node 1, node 2 and node 3). Therefore, the performance network reflects the interrelationship among the three viewpoints reflected by nodes 1, 2, and 3. Node 1 represents the community’s viewpoint that is directly impacted by the transportation system. Node 2 represents the perspective of the transportation service provider whereas node 3 is the transportation user’s perspective. The connection between nodes is directed, indicating the information and/or material transformation from inputs to outputs. From the providers’ point of view, the inputs to the transportation system include different operational costs and the transportation system infrastructure. The costs considered in this research are the system maintenance and administrative costs that the transportation provider typically wishes to minimize. It is also assumed that the provider makes decisions on whether to expand or reconstruct the transportation infrastructure, so it is considered as an input to the provider node 2. The outputs from the provider node 2 include revenue (Revenue I and II in Fig. 3), traffic volume, and level of service (LOS). While collecting revenue (Revenue I) to maintain the transportation system in itself is an objective for the provider, revenue (Revenue II) is also collected as a final output. It is assumed that traffic flow on the roads will result in traffic volume as a consequence of the DSRS and therefore this variable is considered as an output from the provider’s node 2. LOS is included as an output for node 2, because one of the provider’s goals is to provide a certain LOS to the user.

Fig. 3. Three perspectives of the performance network structure.

1144

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

From the community’s point of view, the inputs are the infrastructure, the revenue (Revenue I) from transportation service and the traffic volume. Infrastructure is imposed in the community’s territory, so it is considered as an input for node 1. The traffic volume will result in emissions and accidents in the community, and we assume that part of the revenue (Revenue I) from the DSRS will be used to improve the transit system in the community. Therefore, the traffic volume and revenue (Revenue I) are included as inputs to the community node, and emissions (undesirable output), accidents (undesirable output) and public transportation improvements (desirable output) are the outputs. From the users’ perspective, the fuel cost, travel time and other costs including the reservation fee spent on the trips are considered as inputs by most travelers. These costs are direct expenses. Since this node 3 reflects the user’s perspective, the measurement of the output is considered to be person miles rather than vehicle miles, thus the outputs are person miles traveled and user satisfaction. Among all the variables in the representation of Fig. 3, there are two types of inputs/outputs – intermediate inputs/outputs and initial inputs/final outputs. The final outputs are the outputs that are finally fed into node 4, such as emissions, accidents, and person miles. The intermediate outputs, LOS, traffic volume and revenue, are the outputs from providers’ node 2 and they are also the inputs to nodes1 and 3. 4. Methodology 4.1. Overview of DEA and Network DEA DEA is an analytical technique for measuring the relative efficiency of organizational or production units. It has been widely acknowledged for its strength of (1) capturing multiple inputs and outputs, (2) combining multiple performance dimensions, and (3) computing performance measures that integrate data/information across multiple dimensions and input/output resources (Gattoufi et al., 2004). DEA identifies the empirical frontier that is constituted by the best practice units, and gives an inefficiency/efficiency score to each individual decision making unit (DMU). Since the innovative work by Charnes et al., 1978, DEA has been used in many applications, e.g. education, agriculture, healthcare, and banking industry (Anderson et al., 2007; Cherchye and Van Puyenbroeck, 2007; Cheng et al., 2007; O’Neill et al., 2008). Although DEA may not be one of the most popular tools used in transportation performance measurement, several applications were found in public transit system evaluation (Chu and Fielding, 1992; Karlaftis, 2003, 2004; Kerstens, 1996; Nakanishi and Norsworthy, 2000), airline routes/network performance evaluation (Adler and Golany, 2001; Chiou and Chen, 2006). Other applications in transportation include: evaluating the improvement resulting from TDM programs implemented in 33 worksites where the individual worksite is required to increase vehicle occupancy through the use of rideshare and transit so that a reduction in vehicle trips and vehicle miles travelled is achieved (Nozick et al., 1997), using the DEA approach to find efficient paths in a road network taking into account user mobility (Cardillo and Fortuna, 2000), and the evaluation of public sector investments in Intelligent Transportation Systems (Nakanishi and Falcocchio, 2004). Meanwhile, the network DEA (Färe and Grosskopf, 2000) identifies inefficiency sources embedded in the interactions among various components of the evaluated unit. This approach allows one to further investigate the structure and processes inside the decision making unit, to identify the misallocation of inputs among sub-processes and generate insights about the sources of inefficiency within the DMU. Consequently, this approach allows one to investigate what is transpiring within the ‘‘black box’’ input/output representation of the traditional DEA analysis. Other researchers have extended the network DEA in various directions (Avkiran, 2009; Chen, 2009; Hua and Bian, 2008; Kao, 2009; Löthgren and Tambour, 1999; Prieto and Zofio, 2007). Among them, Sheth et al. (2007) combine the network DEA with goal programming and measured the performance of bus routes whereas Yu and Lin (2008) investigate the efficiency and effectiveness of a group of global railways. 4.2. Radial network DEA model According to Färe and Grosskopf (2000), the network DEA model consists of a set of sub-technologies or activities. In this research, the equivalent sub-technologies or activities are different nodes. The network DEA model is essentially a family of models by formulating a classic DEA model for each node. Assume there are K DMUs and the kth DMU (k = 1, . . . , K) uses I inputs and provides J outputs or intermediate outputs. The formulation will evaluate the efficiency of k0th DMU by computing the efficiency measures hk0, and N sets of intensity variables knk . N equals the number of nodes (not including the dummy nodes). As an illustration, the network DEA model with k = 1, . . . , K observations is written in terms of an output increasing orientation as:

Max hk0 s:t:

0 P xnk i

ð1Þ K X

knk xnk i

n ¼ 1; . . . ; N; i ¼ 1; . . . ; In

ð2Þ

k

k¼1 0 nm ynk P j

0 6 h:k0 ynk j

K X k¼1 K X k¼1

knkm n yj

m ¼ 1; . . . ; N; n ¼ 1; . . . ; N; m–n; j ¼ 1; . . . ; m nJ

ð3Þ

knk ynk j

n ¼ 1; . . . ; N; j ¼ 1; . . . ; J n

ð4Þ

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

0 mn ymk 6 j

K X

k

knkm n yj

1145

m ¼ 1; . . . ; N; n ¼ 1; . . . ; N; m–n; j ¼ 1; . . . ; m nJ

ð5Þ

n ¼ 1; . . . ; N; r ¼ 1; . . . ; Rn

ð6Þ

k¼1 k0 0 qnk 6 r =h

K X

knk qnk r

k¼1

knk P 0 n ¼ 1; . . . ; N; k ¼ 1; . . . ; K xnk i

ð7Þ

nm ynk j

where is the input i to node n for DMU k; the intermediate output j from node m as an input to node n for DMU k; ynk j k the final output j of network from node n for DMU k; m n y j the intermediate output j from node n, meanwhile it is the internal n n input to node m for DMU k; qnk r the undesirable output r from node n for DMU k; I the number of inputs to node n; J the number of desirable outputs from node n; Rn the number of undesirable outputs from node n; nm J n the number of intermediate outputs from node m as an input to node n; mn J m the number of intermediate outputs from node n as an input to node m. Expressions (1)–(7) are considered as the model formulation. The objective is to find a set of intensity variables knk for P P each node. Each node follows the basic DEA technology {yj 6 Kk¼1 kk ykj ; j ¼ 1; . . . ; J; Kk¼1 kk xki 6 xi ; i ¼ 1; . . . ; I; kk P 0; k ¼ 1; . . . ; K}. It allows us to identify an efficient hypothetical DMU that serves as a reference point for each DMU. Expressions (2) and (3) guarantee that the virtual DMU consumes no more of each input and each intermediate input as does DMU k0 at node n. Expressions (4) and (5) ensure that the virtual DMU produces at least as much of each product (i.e. final outputs and intermediate outputs) as does DMU k0 at node n. Compared with Färe and Grosskopf’s model (2000), the difference in this formulation is the inclusion of the undesirable output constraints (6). Expression (7) indicates the non-negative property of the intensity variables. Additionally, we assume that only the final outputs are scaled by factor hk0. Similar to the standard P DEA model, the network model satisfies the variable returns to scale (VRS) assumption, i.e. Kk¼1 knk ¼ 1ð8nÞ. Later in Section k0 k0 5 we report the reciprocal of the output increasing efficiency score, i.e., s ¼ ð1=h Þ 6 1 for reporting consistency purposes. Since the community is assumed to bear the adverse effects of traffic congestion, i.e., emissions and accidents, these outputs are expected to be minimized whereas the other desirable outputs are to be maximized. Efficiency measurement with the incorporation of undesirable outputs in production models has been widely studied. Färe et al. (1989) modified the efficiency measures to allow for asymmetric treatment of desirable and undesirable outputs using the hyperbolic efficiency measurement approach. The hyperbolic efficiency measurement requires solving a nonlinear programming problem. Färe et al. (1989) converted the nonlinear problem to a linear programming problem by taking a linear approximation to the nonk0 0 0 0 linear constraint. qnk 6 Q k. The linear approximation is 2qnk  hk0 qnk 6 Q k. However, Zofio and Prieto (2001) suggested r =h r r P 0 1 0 nk 1 computing the hyperbolic measure by converting the nonlinear constraint to hk0 ðqnk P ð Kk¼1 qnk for computational r Þ r k Þ purposes. However, our analysis has shown that this constraint is still nonlinear with respect to the intensity variables knk. k0 0 For our purposes, the computational benefit from replacing the nonlinear constraint qnk 6 Q k with the constraint r =h P 0 1 0 nk 1 hk0 ðqnk P ð Kk¼1 qnk is very limited, if any. Therefore, in this paper we use the linear transformation proposed by r Þ r k Þ Färe et al. (1989). Nevertheless, other methods of treating undesirable variables are available in the literature, such as indirect ways that transform the values of the undesirable outputs by a monotone decreasing function such that the transformed data can be included as desirable outputs (Scheel, 2001) and an index number approach (Färe et al., 2004). 4.3. Slacks-based network DEA model In DEA, excesses in inputs and shortfalls in outputs are called slacks. Hereafter, the slacks based network DEA model will be referred as the SBMN model. The slacks-based measure of efficiency (SBM) has resulted from the additive models (Cooper et al., 2000). Expanding on the basic SBM model proposed by Tone (2001), Tone and Tsutsui (2009) developed a slacks-based network DEA model. Färe and Grosskopf (2000) use radial efficiency measurement in their network DEA model. In contrast, the SBM model uses a non-radial approach. The radial approach assumes proportionate reduction/increase in inputs/outputs. The SBM model relaxes the proportionate change assumption and aims at obtaining maximum rate of reduction/increase in inputs/outputs. Therefore, the SBM model captures the non-radial slacks directly (Avkiran et al., 2008). Three different models, input-oriented, output-oriented and non-oriented, have been proposed by Tone and Tsutsui (2009) as follows: 

hk0 ¼ minkn ;sn

N X

"

wn 1 

n¼1

1=s

k0

¼ maxkn ;snþ

"

mn sik0 n 1 X n nk0 m i¼1 xi

!#

ð8Þ

rn 1 X skr 0 nþ w 1þ n nk0 r n¼1 r¼1 yr

N X

!#

n

   Pmn ski 0 n n 1 w 1  n¼1 i¼1 nk0 mn xi    k nþ P PN r n sr 0 1 n w 1 þ n nk r¼1 n¼1 r 0

ð9Þ

PN 

qk0 ¼ minkn ;sn ;snþ

yr

ð10Þ

1146

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

Subject to the same constraints:

xnk0 ¼ X n kn þ sn ðn ¼ 1; . . . ; NÞ

ð11Þ

ynk0 ¼ Y n kn  snþ ðn ¼ 1; . . . ; NÞ

ð12Þ

kn P 0; sn P 0; snþ P 0; ð8nÞ

ð13Þ

The intermediate inputs/outputs are called link flows. As regard to the link flow constraints, two cases are proposed1: (1) Discretionary intermediate input/output constraints where the flow between nodes is not restricted.

Z ðl;hÞ kh ¼ Z ðl;hÞ kl ; ð8ðl; hÞÞ;

ð14Þ

(2) Non-discretionary intermediate inputs/outputs constraints where the flow between nodes is restricted by the flow of the DMU being evaluated. ðl;hÞ

Z k0 ¼ Z ðl;hÞ kh ð8ðl; hÞÞ;

ð15Þ

ðl;hÞ

Z k0 ¼ Z ðl;hÞ kl ð8ðl; hÞÞ:

where h is the input-oriented efficiency; s the output-oriented efficiency; q the non-oriented efficiency; wn the relative weight of node n corresponding to its importance. It is assumed that all nodes have the same importance in the example of Section 5. xn the external input vector to node n; yn the final output vector from node n; z(l,h) the intermediate products produced from node l and consumed by node h; sn is the input excess vector in node n; sn+ is the output shortages vector in node n; kn is the intensity variable vector of node n; mn the number of inputs in node n; and rn is the number of outputs in P node n.Similarly, Kk¼1 knk ¼ 1ð8nÞ if variable returns to scale holds. To take into account the undesirable output, this paper adopts the approach implemented in the DEA-Solver Pro software.2 In the presence of undesirable outputs, denoted by b, a DMU is efficient if there is no vector (x, y, b) e P such that x0 P x, y0 6 y, b0 P b with at least one strict inequality where P is the production possibility set. In accordance with this definition, the SBMN (non-oriented model) is modified as follows:

   0 Pmn snk i 1  m1n i¼1 nk0 xi    ¼ min nk ;b P P PN n1 snk0 ;g s sn2 sr 0 r n 1þ 1 w þ r¼1 nk0 ;g r¼1 nk0 ;b n¼1 sn PN



qk0

n n¼1 w

yr

ð16Þ

yr

Subject to:

xk0 ¼ Xk þ s yk0 ;g ¼ Yk  sg yk0 b ¼ Yk þ sb

ð17Þ

s ; sg ; sb ; k P 0 The superscript ‘‘g’’ indicates desirable outputs, ‘‘b’’ indicates undesirable outputs, and sn1, sn2 are the number of desirable and undesirable outputs for node n respectively where sn = sn1 + sn2.

5. An illustrative example3 Because of the limited number of demand scenarios (DMUs) available from the microscopic simulation, the DEA network has been simplified in our example (Fig. 4), keeping representative variables of the network of Fig. 3. In addition, since the simulation is conducted on the same transportation network, the infrastructure is fixed, thus it not included in Fig. 4 and in the following formulation. The network DEA model is written in terms of an output increasing perspective since we are interested in the degree to which desirable outputs are maximized as:

Max hko

ð18Þ

1 In the example provided in Section 5, the discretionary case is assumed since we did not wish to restrict the flow between nodes. It is assumed that the link flow or the intermediate products can be increased or decreased in the optimal solution of the linear programming formulation. 2 (http://www.saitech-Inc.com/index.asp). 3 The DEA, network DEA, and slacks-based network DEA were programmed using Premium Excel Solver.

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1147

Fig. 4. The simplified network.

Subject to: Node 1 (community’s perspective): p k0 c y v mt

P

K X

k

kck  pc y v mt

ð19Þ

kck yck e

ð20Þ

k¼1

0 yck e =h 6

K X k¼1

Node 2 (providers’ perspective): K X

pk

xcos0 t P

kpk xpk cos t

ð21Þ

k¼1

p k0 u y as

6

K X

k

kpkpu y as

ð22Þ

k¼1

p k0 c y v mt

K X

6

kpkpc y v mt

k

ð23Þ

kpk ypk r

ð24Þ

k¼1

0 6 h:ypk r

K X k¼1

Node 3 (users’ perspective):

0 xuk P f

K X

kuk xuk f

ð25Þ

kuk xuk tt

ð26Þ

k¼1

0 xuk tt P

K X k¼1

p k0 u y as

P

K X k¼1

k

kukpu y as

ð27Þ

1148

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

0 h:yuk pm 6

K X

kuk yuk pm

ð28Þ

k¼1 K X

knk ¼ 1ðn ¼ u; p; cÞ

ð29Þ

i¼1

k

where pu y v mt is the vehicle miles traveled (vmt), intermediate output from the provider node and intermediate input to the community node for demand scenario k; yck e the emission, undesirable output from the community node for demand scenario k; xpk cos t the system operation cost (maintenance cost and administrative cost) incurred by the system provider for dek mand scenario k; pu y as the average speed, intermediate output from the provider node and input to the user node for demand scenario k; ypk the total revenue obtained from the reservation system, an output of the provider node for demand scenario k; r xuk the fuel consumption, input consumed by transportation users for demand scenario k; xuk tt the average travel time input f used by transportation users for demand scenario k; yuk pm the person miles traveled (PMT), final output from the users’ perspective for demand scenario k; and kpk ; kuk ; kck are non-negative intensity variables associated for each node. Expressions (19) and (20) are associated with the community node, (21)–(24) are associated with the provider node, and (25)–(28) are associated with the user node. The model allows us to identify an efficient hypothetical DMU that serves as a reference point for each DMU. VRS is assumed here. However, the constant returns to scale (CRS) results are presented in some of the following sections for comparison purposes. Furthermore, the above formulation has an output orientation. We also modified this formulation to compute the efficiency scores from an input orientation (see Appendix B). These results are reported in Section 5.3. Finally, input and output oriented VRS models were run where only the external inputs and final outputs were considered, i.e., the node network representation was assumed away. These models are labeled as aggregate models in the description of the results in Section 5.5. 5.1. Data description The unit of analysis is the demand scenario with the DSRS implemented in the transportation system. Data associated with the demand scenarios are obtained from the micro simulation model. The scenarios were varied in terms of the total demand level (i.e. number of vehicles per control period), the reservation policies (i.e. the weights assigned to the people throughput and revenue in the objective function of the optimization model) and the inherent uncertainty of the traffic assignment and the traffic flow in the simulation (Table 1). The demand level varies from 6000 to 7000 (vehicles/control period) and is selected according to the transportation network size of the traffic simulation model. The relative importance (and consequently the weights of the DSRS) associated with the people throughput and revenue in the original optimization model of the DSRS is arbitrarily assigned due to the lack of practical references. Table 1 shows that DMU 6–DMU 16 have the same demand level and weights. They differ due to the stochastic nature of the traffic assignment and the traffic flow. Altogether, there are 28 scenarios, thus 28 DMUs. Operational cost is assumed constant for all 28 DMUs. Table 2 provides the statistical summary for the data set. 5.2. Radial network model vs. separate DEA models for each node In this section, the analysis is based on the assumptions of output maximization and variable returns to scale. This section focuses on exploring the relationship between the nodes and the entire network. To show how the node efficiency affects the network efficiency, we first ran standard DEA models for each node separately (Fig. 5), and then ran the radial network DEA

Table 1 Demand scenarios. DMU

Demand level (# of vehicles)

Weights

DMU

Demand level

Weights

1 2 3 4 5 6 7 8 9 10 11 12 13 14

6000 6200 6400 6600 6800 7000 7000 7000 7000 7000 7000 7000 7000 7000

3 3 3 3 3 3 3 3 3 3 3 3 3 3

15 16 17 18 19 20 21 22 23 24 25 26 27 28

7000 7000 7000 7000 7000 7000 7000 7000 7000 7000 7000 7000 7000 7000

3 (3:1) 3 (3:1) 1 (1:1) 2 (2:1) 3 (3:1) 4 (4:1) 5 (5:1) 8 (8:1) 10 (10:1) 20 (20:1) 30 (30:1) 40 (40:1) 50 (50:1) +1 (1:0)

(3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1) (3:1)

1149

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159 Table 2 Summary statistics of the inputs and outputs.a

Mean Minimum Maximum Standard Deviation STD (%) a

Fuel cost (gal)

Travel time (s)

Revenue ($)

Avg. speed (mph)

Emission (g)

Vehicle miles traveled (mile)

Input to user node 3

Input to user node 3

Output from provider node 2

Link from provider node 2 to user node 3

Output from community node 1

Link from provider node 2 to community node 1

Person miles traveled (mile) Output from user node 3

349.12 299.91 370.65 16.46

216.11 176.96 268.83 18.63

22251.7 20401.7 23679.8 526.56

12.83 11.38 13.21 0.35

34807.9 29901.1 36954.13 1639.837

4179.25 3743.16 4366.56 151.25

14325.75 11385.04 16862.95 1562.631

4.71%

8.62%

2.37%

2.72%

4.71%

3.62%

10.91%

Operation cost for the community node is assumed constant for all demand scenarios.

Fig. 5. Separate/individual DEA models for the nodes. Note: For the community node, in addition to emissions (undesirable output) a constant good output value is assumed for all DMUs (see Table 3).

model (Section 4.2). According to the efficiency scores obtained from the models for each node separately and the radial network DEA model (Table 3), the following observations are obtained: (1) if one of the nodes is efficient (technical efficiency), then the radial network DEA representation is efficient (technical efficiency); (2) the radial network efficiency score equals the efficiency score associated with one of the nodes; (3) for this application, the network efficiency is dominated by the provider node. What this means is that the provider node is the node that is efficient most of the time when efficiency is computed considering the provider node as a separate node and the radial network efficiency is determined by this node when this happens (This is consistent with the results depicted in Table 3); in general, the radial network representation is dominated by the most efficient node as indicated by the separate DEA models; (4) in this application, a DMU is mix efficient (efficiency score of one and zero slacks, see Table 4) if it is efficient when considering the provider and user nodes using separate DEA models (Cooper et al., 2000); (5) the standard deviation in Table 2 suggests that the variation of the input and output data across the 28 DMUs is not large. Therefore, it is not surprising that the efficiency scores do not differ by that much. For the radial network model, each node has one set of intensity variables and each node has its own reference set. The reference set for DMU k0 at node n is Rnk0 = {j|knj > 0, where j is the index for the DMUs in the sample}. For the provider node, DMUs 2, 5, 10, 14, 18, 20, 25, and 28 are considered as reference peers for the inefficient DMUs (Fig. 6 depicts the peers for each node when using the radial network DEA model). Among them, DMUs 5, 18 are the ones most frequently encountered. DMU 18 has the highest vehicle miles travelled (VMT) whereas DMU 5 has the highest revenue. Even if DMU 5 has a relatively low VMT, from a maximizing revenue point of view, it outperforms others in terms of the provider’s perspective. Other DMUs could improve performance through increasing revenue as their peer DMU 5 suggests. DMU10 has high revenue (3rd). DMU14 has moderate revenue (10th), but has very high average speed (2nd). For the user node, DMUs 16 and 28 are the reference peers for all the other DMUs (Fig. 6). DMU 28 has the highest person miles traveled whereas DMU 16 has the second highest person miles traveled. All these reference DMUs for both the provider’s and user’s nodes are also technically

1150

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

Table 3 Node efficiency scores using separate DEA models and radial network efficiency scores (VRS output-oriented).a DMU

Provider

User

Communityb

Network

DMU

Provider

User

Community

Network

1 2 3 4 5 6 7 8 9 10 11 12 13 14

0.973 1.000 0.981 0.987 1.000 0.974 0.978 0.973 0.973 1.000 0.965 0.978 0.998 1.000

0.922 1.000 0.864 0.965 1.000 1.000 0.808 0.844 0.857 0.880 0.908 0.992 0.800 0.931

0.925 1.000 0.950 0.956 0.956 0.902 0.893 0.882 0.887 0.908 0.895 0.907 0.900 0.898

0.973 1.000 0.981 0.987 1.000 1.000 0.978 0.973 0.973 1.000 0.965 0.992 0.998 1.000

15 16 17 18 19 20 21 22 23 24 25 26 27 28

0.964 0.985 0.997 1.000 0.978 1.000 0.993 0.989 0.996 1.000 1.000 0.986 0.992 1.000

1.000 1.000 0.962 0.870 0.951 0.795 0.847 1.000 0.941 0.881 0.951 1.000 0.913 1.000

0.904 0.900 0.885 0.867 0.893 0.867 0.869 0.871 0.873 0.865 0.864 0.869 0.865 0.857

1.000 1.000 0.997 1.000 0.951 1.000 0.993 1.000 0.996 1.000 1.000 1.000 0.992 1.000

a

The values in the table are the inverse of the output increasing efficiency scores that are provided for consistency purposes. These values are based on the inverse of the input reducing variable returns to scale efficiency scores where in addition to emissions (undesirable output) a constant good output value is assumed for all DMUs. b

Table 4 Slacks in the radial network model (VRS output-oriented). DMU

Network Efficiency ( mix efficient)

(Input) cost

(Output) revenue

(Input) fuel

(Input) travel time

(Output) person miles

(Output) emission

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

0.973 1.000 0.981 0.987 1.000 1.000 0.978 0.973 0.973 1.000 0.965 0.992 0.998 1.000 1.000 1.000 0.997 1.000 0.951 1.000 0.993 1.000 0.996 1.000 1.000 1.000 0.992 1.000

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 603.62 0 0 0 0 0 0 0 0 0 163.37 0 0 0 0 0 230.06 0 0 0 291.96 0 0

2.15 0 0 0 0 0 0 4.42 0 0 0 0 0 0 0 0 4.72 0 0 0 0 0 0 0 0 0 0 0

10.58 0 36.84 6.58 0 0 19.78 0 21.60 10.35 1.69 4.54 27.00 7.44 0 0 8.41 34.63 19.01 28.31 10.63 0 19.46 1.21 5.41 0 21.98 0

0.00 0 655.35 0 0 0 1704.07 1884.33 761.59 1102.29 796.82 0 1791.69 522.91 0 0 590.38 1323.60 174.69 2166.45 2094.44 0 913.01 1832.93 662.55 0 1023.64 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

efficient as far as the network is concerned. However, for the community node, the results show that some DMUs do consider some network inefficient (network efficiency scores are greater than one) DMUs as their reference peers. This is because the network efficiency is mainly dominated by the provider node.4 Therefore, even though DMUs that are considered as peers by the community node are more efficient from the community perspective (and consequently can be considered as a peer in that perspective), they still have inefficient network scores because of their performance associated with the provider node. 4 Dominance of the provider node might depend on the fact that more outputs factors are specified for this node and this in turn would impact the radial network efficiency score. In other words, the radial network efficiency score is linked to the relatively efficient node whose efficiency score is determined in part by its number of inputs/outputs. However, as Table 3 indicates there are cases where the customer node determines the efficiency of the network indicating that both the quantity and quality of the inputs and outputs affect the relative efficiency of the node and the network.

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1151

Fig. 6. Reference DMUs in the radial network model (variable returns to scale output-orientation).

In this example, only DMU 28 is mix efficient under both VRS and CRS (see Appendix A for CRS results). DMU 28 represents the demand scenario with maximum weight placed on people throughput in the objective function of the DSRS optimization formulation. In this scenario, the decision maker decides whether to accept the incoming request based on the sole goal of maximizing the total number of people throughput, and does not take into account revenue maximization. DMU 28 has the lowest revenue but it has the highest person miles among the 28 DMUs. The people throughput is reflected by node 3 (User Node). Therefore, for that node, 18 out of the 28 DMUs refer to DMU 28 as their peer (Fig. 6). This shows that the reservation system may consider the instance of DMU 28 as a base case and then the decision maker can evaluate how far current circumstances deviate from this base case. Table 4 shows that the slacks mostly occur for the user node for travel time and person miles traveled5 in the radial network model. The radial efficient but mix inefficient DMUs, such as DMUs 10, 14, 18, 20, can be improved by reducing the travel time and/or increasing the person miles traveled without compromising the other inputs and outputs. Additionally, there are no operational cost slacks associated with the provider node. The inefficiency associated with provider node is mainly radial inefficiency, while the inefficiency in the user node is mainly mix inefficiency.

5.3. Radial network model: output maximization vs. input minimization orientations Comparison between input and output oriented models shows that the output oriented model provides higher discriminating power among the 28 DMUs (Fig. 7). Especially under the VRS assumption, all 28 DMUs of the input oriented model are technically efficient. We suspect that this result is driven in part by the assumption that all DMUs have the same operational 5

This statement is also supported by the results of the SBMN model presented in Section 5.4.

1152

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

Fig. 7. Radial network DEA: output vs. input orientations (variable returns to scale). Notes: For the input oriented M2 model, only travel time and fuel cost are minimized. In the input oriented model, operational cost, travel time and fuel cost are minized. The reciprocal of the output oriented efficiency scores for presentation purposes have been considered for this figure.

Fig. 8. Efficiency comparison of slacks based measurement network (SBMN) models (constant (CRS) versus variable returns to scale (VRS)). Note: SBM_I – Slacks based model input oriented; SBM_0 –slacks based model output oriented; SBM_NO – slacks based model non-oriented.

cost for the provider node. Therefore, we consider an alternative input oriented model (M2) where the operational costs are not minimized but considered constant and only the inputs to the user node are minimized. Fig. 7 shows that the results from the M2 formulation discriminate more among the DMUs. This further suggests that if input minimization is desirable for the system, the decision maker should emphasize reducing user’s travel time and fuel consumption when there is no or very little variability associated with the operational cost associated with running the downtown space reservation system. This also emphasizes the importance of the user’s perspective in assessing the network performance for this specific application.

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1153

5.4. Slacks based network model Comparing the three SBMN models (Fig. 8) under both VRS and CRS cases, the non-oriented model provides a relatively low efficiency, and DMUs are closer to being efficient in the input oriented model. This suggests that less inefficiency exists in the input excesses. In other words, the inefficiency results from the output shortages. The SBMN formulation implies that the (in)efficiency of the three nodes is averaged in the network model. The SBMN efficiency score is determined jointly by a weighted average of the three nodes. In contrast, in the radial network DEA model, the network efficiency of a DMU is determined by one of the three nodes, and the provider node dominates the network. What this means is that for each DMU, the network efficiency score refers only to the relatively efficient node in the network and ignores the inferior performance of the other nodes. Compared with this, the slacks based measurement network model considers average performance of all nodes. Therefore, dominance is different when comparing the two models. Nevertheless, Table 5 suggests that the SBMN network efficiency scores have a higher correlation with the user node efficiency, and this indicates the dominance of the user node in the SBMN formulation even when we assume all nodes have the same weights in the objective function (9). What is meant by node efficiency here is the calculation of the efficiency score for each node separately using the slacks based approach (Tone and Tsutsui, 2009). This is consistent with the slack results found in the radial network DEA model. It shows that the slacks are mainly from the travel time (input to user node) and person miles (output from user node). In the SBMN formulation, the network efficiency scores have the least correlation with the provider node.

5.5. Radial network model vs. slacks based network (SBMN) model Compared to the radial network model, the SBMN models have a higher discriminating power (Fig. 9). More DMUs are efficient in the radial network model. This indicates that if the proportional (radial) reduction/increase assumption is relaxed, some DMUs are inefficient, whereas they are efficient in the radial network model. Comparing the reference sets in the radial network model (Fig. 6) and the SBMN models (Fig. 10), DMUs 2 and 5 are peer DMUs for the inefficient DMUs for the provider node in the radial network DEA models. In the SBMN models, DMU 2 appears in the reference sets of all the three nodes with much higher frequency. Although the frequency with which a DMU appears as a reference DMU may be different in the two different network DEA approaches, the majority of the referenced DMUs are the same. In the SBMN models, the reference peers can be inefficient themselves, but the reference peers should be efficient at least at one node.

Table 5 Correlation of the slacks based measurement network (SBMN) efficiency and slacks based model (SBM) node efficiencya scores (VRS).

Network P-Node U-Node C-Node

Non-oriented

Output-oriented

1 0.49 0.85 0.65

1 0.56 0.82 0.63

a What is meant by SBM node efficiency here is the calculation of the efficiency score for each node separately using the slacks based approach (Tone and Tsutsui, 2009).

Fig. 9. Efficiency comparison of aggregate and network models (variable returns to scale). Notes: Aggregate-Is where only the external inputs and final outputs are considered, i.e., the node network representation was assumed away; network is the radial network model; SBM is the slacks based model. The reciprocal of the output oriented efficiency scores for presentation purposes have been considered for this figure.

1154

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

Fig. 10. Reference sets in slacks based measurement network SBMN models (variable returns to scale). Note: SBM_0 – slacks based model output oriented; SBM_NO – slacks based model non-oriented.

Table 6 Efficiency performance representation of DMU 2, 5 and 28 (VRS). DMU

2 Eff.

Network output oriented Network input oriented SBMN non-oriented

SBMN output oriented

SBMN input oriented

Aggregate (output) Aggregate (input)

Network P-Node U-Node Network P-Node U-Node Network P-Node

1 1 1 1 1 1 1 1

U-Node C-Node

5 Ref. set

Eff.

28 Ref. set

Slacks emission

Eff.

2

1 1 1 1 1 1 0.9905 1

1 1

2 2

1 0.9720

Network P-Node

1 1

2

0.9905 1

U-Node C-Node

1 1

2 2

1 0.9720

Network P-Node U-Node C-Node

1 1 1 1

2 2 2

1 1 1 1

5 5 5

1 1 1 1

28 28 28

1 1

2 2

1 1

5 5

1 1

28 28

2 2 2 2

1 1 1 1 1 1 0.9151 0.8713

Ref. set

5 5 5 5 5 5 2(0.7) 20(0.3)

930

0.9151 0.8713

5 5 2(0.7) 20(0.3)

1 0.8844

930

1 0.8844

Slacks revenue

Slacks emission

28 28 28 28 5(0.8), 15(0.2) 28 2(0.6) 20(0.4)

3013

5(0.8), 15(0.2) 28 2(0.6) 20(0.4)

3013

4770

4770

Let us take DMUs 2, 5, 28 as representative examples (Table 6). DMU 2 is efficient in all of the models for all of the three perspectives (nodes). It is a reference point for most of the other DMUs. The input and output measures associated with the demand scenario show that this scenario has a low demand level (k = 6200), relatively high revenue from the provider’s perspective, low fuel consumption and travel time from the user’s perspective, and low emission from the community perspective. Therefore, in this example, this DMU is the ‘‘global’’ efficient one and can be considered as a benchmark for the other DMUs. DMU 5 is efficient in the SBMN input oriented model and the radial network DEA models, but inefficient in the SBMN output oriented and non-oriented models. This indicates that the inefficiency comes from the output slacks. The node efficiency further indicates that the slack inefficiency is from the community node and is associated with emissions. This unit can be improved by reducing its emission by 930 g. DMU 5 is efficient considering both orientations in the radial network DEA models and the aggregate models. In this case, the network models provide as much insight about the demand scenario as the aggregate models do. The SBMN models further provide information about the source of inefficiency associated with the slacks. The conclusions for DMU 28 are similar to the conclusions reached for DMU 5. Its inefficiency in the SBMN models results from an output shortage of revenue ($3013) and an emission excess (4770 g). 5.6. Concluding remarks from the example In the radial network DEA, the network efficiency is dominated by the provider node. This results from the model structure. The provider node is the center of the network. It provides intermediate inputs to the community node and user node. This structure determines that the performance in the provider node inevitably impacts the performance of the other two

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1155

nodes. However, the SBMN efficiency is constructed as a weighted average of the node efficiency. This average provides alternative information about the relationship of the network performance in relation to the node performance. Consequently, for the decision maker, the radial network DEA approach and the SBM network DEA approach provide different types of information. As stated earlier, the dominance is different when considering both approaches given that the radial network efficiency score refers only to the relatively efficient node in the network and ignores the inferior performance of the other nodes whereas the slacks based network measure considers the average performance of all nodes. According to this information, the decision maker may focus on different interventions so as to improve system performance. For instance, in our example, the original network DEA will lead the decision maker to focus more on the provider’s perspective, while the SBM network model will lead decision maker to focus on the user’s node. The network DEA structure should be indicative of the underlying transportation network structure and of the transportation processes that are taking place. If the DEA framework somewhat inaccurately reflects the real transportation system, then the performance evaluation will be compromised. Additionally, because the evaluation of a DMU in DEA is relative to all the other DMUs, it is highly data dependent. In our example we did not have all the possible demand scenarios for the DSRS since we were limited by the number of micro simulation experiments we could run. Therefore, the DMUs used in the example represent a subset of the production possibility set. Because of this limitation we cannot generalize the findings associated with specific input and output variables. However, this limitation will be alleviated once a more comprehensive set of simulation experiments is executed. Furthermore, the methodology proposed provides a framework for performance assessment that can generate important insights both in assessing the performance of existing or of future transportation systems as is the case with the DSRS.

6. Research conclusions and future directions This research contributes to the extant transportation and performance measurement literature as follows: (1) The application of network DEA in transportation performance measurement is relatively immature in the sense that few researchers have attempted to focus on these applications. This research presents a demonstration of a novel application of network DEA for a transportation system where a TDM strategy, i.e., the DSRS is being evaluated but has yet to be implemented. (2) This paper combines the network DEA approach with traffic simulation in performance measurement. It utilizes the outputs from the microscopic traffic simulation models as inputs to the DEA model. Thus, the networkDEA model complements the micro level simulation performance evaluation by accounting for macro level performance measurement considerations. (3) Furthermore, for validation and completeness purposes, the current research compares two different types of network DEA models, the original network DEA model proposed by Färe and Grosskopf (2000) and the slacks based network DEA model proposed by Tone and Tsutsui (2009). To the authors’ knowledge, this is the first attempt to compare the two approaches. (4) Finally, we connect the DEA performance measurement approach with the future design of the DSRS. The illustrative example presented in this paper is used to suggest how the performance measurement assessment can discover potential improvement directions and how these can be included as requirements into the design of the DSRS at an early stage.

6.1. Bridging optimization, micro simulation and network DEA This research bridges the DSRS optimization model, the micro simulation approach and the network DEA approach. Optimization and simulation are popular methods used in system engineering design. The optimization model is the foundation of the DSRS research and constitutes the actual decision making approach that enables the decision maker to decide whether to accept or reject a trip reservation request. It is at the core of the DSRS design. However, the optimization model itself cannot convey information, such as which traffic flow conditions are best suited for the DSRS, whether the design of the system meets stakeholders’ requirements, and how the DSRS influences performance, etc. Therefore, once the DSRS is implemented an additional evaluation should be done to determine whether requirements and performance goals of the DSRS are being met. Simulation is one of the most popular tools used by transportation professionals. It has been used to test and analyze the DSRS (Zhao et al., 2010b). The simulation model provides various transportation measures (e.g. travel time, average delay, etc.) and helps the decision maker appreciate the system from a transportation engineering perspective. However, additional performance measurement and system design issues need to be addressed beyond the simulation paradigm. First, it is not the absolute representation of performance that matters, but the idea of relative performance that is important. For example, one might be more interested in how much performance can be improved when compared to best practices. Moreover, the simulation does not directly tell us how the key performance measures interact with each other. The current paper addresses these issues with a comprehensive performance measurement approach that provides a single index as representative of the overall transportation system efficiency and identifies the sources of inefficiency. In conclusion, the optimization model represents the system that was designed; the simulation and the network DEA models are the supporting approaches that provide an evaluation of this system design. The two evaluation approaches are complementary. For the DSRS, the simulation approach supports the network DEA model by providing data, and the

1156

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

network DEA performance measurement complements the simulation model by taking into account the key perspectives that are impacted by the potential implementation of the DSRS. 6.2. System design and social welfare implications The current research provides an illustrative demonstration that shows how the network DEA assesses the DSRS or transportation systems in general. The network DEA approach helps the decision maker understand the system that is being evaluated by opening the classic DEA transformation ‘‘black box’’. This enables decision makers to locate the sources of inefficiency more accurately. It helps detect potential system design improvements and facilitates the definition of requirements into the design at the early stage. For instance, if the network DEA model shows that inefficiency is derived mainly from the user node, the decision maker will need to put more effort on the user’s perspective, such as improving the people throughput via a pricing policy adjustment. In addition, the research provides valuable insights concerning the social welfare impact of the DSRS. At the beginning of the DSRS development (Zhao et al., 2010a,b), the design of the system was oriented by the goal of mitigating congestion. It did not attempt to relate this goal to the producer, user and community perspectives. This means that the original optimization model, which is at the core of the DSRS was not elaborated in this way. The approach provided in this paper provides an assessment of the social welfare associated with the DSRS insofar that the demand scenarios obtained from the original DSRS optimization approach can be evaluated from the user’s and community’s perspectives using multiple and diverse measures. 6.3. Future research directions Taking into account the fact that limited network DEA applications are available for transportation performance measurement, the following directions are recommended: (1) The research can be expanded by conducting further analysis that will generate additional data that can come from different demand distributions. This will allow the evaluation of the transportation network as it responds to varying demand scenarios using the DEA based performance measurement framework. With the help of the evaluation, the decision maker will know under which demand scenarios the DSRS will work better, and to what extent. In addition, the system designer could improve the design by considering the DSRS as a mechanism to adjust travel demand.

Table A Efficiency scores and slacks in the radial network model (constant returns to scale output-oriented). DMU

Radial network efficiency

(I) Cost

(O) Revenue

(I) Fuel

(I) Travel t.

(O) PM

(O) Emission

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

1.0208 1.0000 1.0190 1.0127 1.0000 1.0271 1.0218 1.0279 1.0281 1.0000 1.0356 1.0216 1.0022 1.0000 1.0368 1.0000 1.0026 1.0000 1.0218 1.0000 1.0068 1.0105 1.0042 1.0000 1.0000 1.0142 1.0077 1.0000

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 163.37 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

34.69 60.99 53.34 33.15 16.01 18.96 52.63 60.92 0.00 45.99 0.00 0.00 58.14 0.00 17.19 0.00 0.00 0.00 0.00 62.78 50.07 5.33 0.00 0.00 16.46 19.39 0.00 0.00

30.54 37.41 69.20 34.14 38.87 28.93 51.13 33.63 21.95 37.96 2.06 8.70 61.69 7.96 70.42 0.00 5.64 34.87 19.09 65.64 40.20 0.54 25.21 1.38 15.14 0.00 22.07 0.00

0.00 0.00 0.00 0.00 0.00 1198.57 0.00 0.00 1403.69 0.00 1483.41 16.57 0.00 1488.62 719.31 0.00 805.04 1769.03 324.98 0.00 0.00 0.00 726.01 2142.50 0.00 440.89 1185.99 0.00

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1157

(2) The analysis in Section 5 shows the transportation system performance with the DSRS under different demand scenarios. Future research can collect data from another set of demand scenarios where the DSRS is not implemented, and run the network DEA model with the new data. In this case, there will be two different frontiers – the frontier corresponds to the transportation system with the DSRS and another frontier without the DSRS. The comparison of the two frontiers would provide insights about how the DSRS impacts the transportation system. (3) Future research can adopt this performance measurement framework to evaluate other travel demand management strategies, such as intelligent parking reservation systems, and congestion pricing. It will be critical to validate the performance measurement results with data of systems that have been implemented as would be the case with congestion pricing. (4) The performance network developed in this research is one of the many ways to represent the transportation system and the formulation and results obtained are contingent upon the understanding of the transportation network structure and the underlying transportation processes. Future research may provide alternative performance measurement formulations (e.g., dynamic performance representations) that align better with the transportation systems they seek to evaluate.

Acknowledgements This research has been supported by the National Science Foundation (Project #0527252). Any opinions, conclusions, and/or findings are those of the authors and do not necessarily reflect the views of NSF. Appendix A Table A. Appendix B Input. oriented original network model

Min hko

ð30Þ

Subject to: Node 1 (community’s perspective): K X

k

h:pc y v0mt P

k

kck  pc y v mt

ð31Þ

k¼1

0 yck 6 e

K X

kck yck e

ð32Þ

k¼1

Node 2 (providers’ perspective):

pk

h:xcos0 t P

K X

kpk xpk cos t

ð33Þ

k

ð34Þ

k¼1

p k0 u y as

6

K X

kpkpu y as

k¼1

p k0 c y v mt

6

K X

k

p kpk p c y v mt

ð35Þ

k¼1

0 6 ypk r

K X k¼1

kpk ypk r

ð36Þ

1158

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

Node 3 (users’ perspective): 0 h:xuk P f

K X

kuk xuk f

ð37Þ

kuk xuk tt

ð38Þ

k¼1

0 h:xuk tt P

K X k¼1

p k0 u y as

P

K X

k

kukpu y as

ð39Þ

k¼1 K X

kuk yuk pm

ð40Þ

knk ¼ 1 ðn ¼ u; p; cÞ

ð41Þ

0 yuk pm 6

k¼1 K X i¼1 k

where pc y v mt is the vehicle miles traveled (vmt), intermediate output from the provider node and intermediate input to the pk community node for demand scenario k; yck e the emission, output from the community node for demand scenario k; xcos t the system operation cost (maintenance cost and administrative cost) incurred by the system provider for demand scenario k; p k pk u y as the average speed, intermediate output from the provider node and input to the user node for demand scenario k; yr the total revenue obtained from the reservation system, an output of the provider node for demand scenario k; xuk the fuel conf sumption, input consumed by transportation users for demand scenario k; xuk tt the average travel time used by transportation users for demand scenario k; yuk pm the person miles traveled (PMT), final output from users’ perspective for demand scenario k; and kpk ; kuk ; kck are non-negative intensity variables associated for each node. References Adler, N., Golany, B., 2001. Evaluation of deregulated airline networks using data envelopment analysis combined with principal component analysis with an application to Western Europe. European Journal of Operational Research 132, 260–273. Anderson, T.R., Daim, T.U., Lavoie, F.F., 2007. Measuring the efficiency of university technology transfer. Technovation 27 (5), 306–318. Avkiran, N.K., Tone, K., Tsutsui, M., 2008. Bridging radial and non-radial measures of efficiency in DEA. Annals of Operations Research 164 (1), 127–138. Avkiran, N.K., 2009. Opening the black box of efficiency analysis: an illustration with UAE Banks. Omega 37, 930–941. Cambridge Systematics, 2005. Traffic Congestion and Reliability: Trends and Advanced Strategies for Congestion Mitigation. Cardillo, L., Fortuna, T., 2000. A DEA model for the efficiency evaluation of nondominated paths on a road network. European Journal of Operational Research 121, 549–558. Charnes, A., Cooper, W.W., Rhodes, E., 1978. Measuring the efficiency of decision making units. European Journal of Operational Research 2, 429–444. Chen, C.-M., 2009. A network-DEA model with new efficiency measures to incorporate the dynamic effect in production networks. European Journal of Operational Research 194 (3), 687–699. Cheng, E.W.L., Chiang, Y.H., Tang, B.S., 2007. Alternative approach to credit scoring by DEA: evaluating borrowers with respect to PFI projects. Building and Environment 42 (4), 1752–1760. Cherchye, L., Van Puyenbroeck, T., 2007. Profit efficiency analysis under limited information with an application to German farm types. Omega International Journal of Management Science 35 (3), 335–349. Chiou, Y.-C., Chen, Y.-H., 2006. Routed-based performance evaluation of Taiwanese domestic airlines using data envelopment analysis. Transportation Research Part E 42, 116–127. Chu, X., Fielding, G.J., 1992. Measuring transit performance using data envelopment analysis. Transportation Research Part A 26A (3), 223–230. Cooper, W.W., Seiford, L.M., Tone, K., 2000. Data Envelopment Analysis: A Comprehensive Text with Models, Applications, References and DEA-Solver Software. Kluwer Academic Publishers, Boston/Dordrecht/London. Falcocchio, J.C., 2004. Performance measures for evaluating transportation systems. Transportation Research Record: Journal of the Transportation Research Board, No. 1895, 220–227. Färe, R., Grosskopf, S., 2000. Network DEA. Socio-Economic Planning Sciences 34, 35–49. Färe, R., Grosskopf, S., Hernandez-Sancho, F., 2004. Environmental performance: an index number approach. Resource and Energy Economics 26, 343–352. Färe, R., Grosskopf, S., Lovell, C.A.K., Car, P., 1989. Multilateral productivity comparisons when some outputs are undesirable: a nonparametric approach. The Review of Economics and Statistics 71, 90–98. Gattoufi, S., Oral, M., Kumar, A., Reisman, A., 2004. Content analysis of data envelopment analysis literature and its comparison with that of other OR/MS fields. Journal of Operational Research Society 55, 911–935. Hua, Z., Bian, Y., 2008. Performance measurement for network DEA with undesirable factors. International Journal of Management and Decision Making 9 (2), 141–153. Jeon, C.M., Amekudzi, A., 2005. Addressing sustainability in transportation systems: definitions, indicators, and metrics. Journal of Infrastructure Systems 11 (1), 31–50. Kao, C., 2009. Efficiency decomposition in network data envelopment analysis: a relational model. European Journal of Operational Research 192 (3), 949– 962.

Y. Zhao et al. / Transportation Research Part E 47 (2011) 1140–1159

1159

Karlaftis, M.G., 2003. Investigating transit production and performance: a programming approach. Transportation Research Part A 37, 225–240. Karlaftis, M.G., 2004. A DEA approach for evaluating the efficiency and effectiveness of urban transit systems. European Journal of Operational Research 152, 354–364. Kerstens, K., 1996. Technical efficiency measurement and explanation of French urban transit companies. Transportation Research Part A 30 (6), 431–452. Litman, T., 2008. Well Measured: Developing Indicators for Comprehensive and Sustainable Transport Planning. Victoria Transport Policy Institute. Löthgren, M., Tambour, M., 1999. Productivity and customer satisfaction in Swedish pharmacies: a DEA network model. European Journal of Operational Research 115, 449–458. Nakanishi, Y.J., Falcocchio, J.C., 2004. Performance assessment of intelligent transportation systems using data envelopment analysis. Economic impacts of intelligent transportation systems: innovations and case studies. Research in Transportation Economics 8, 181–197. Nakanishi, Y.J., Norswothy, J.R., 2000. Assessing efficiency of transit service. In: Paper Presented at the 2000 IEEE International Engineering Management Society Conference. Nozick, L.K., Borderas, H., Meyburg, A.H., 1997. Evaluation of travel demand measures and programs: a data envelopment analysis approach. Transportation Research Part A 32 (5), 331–343. O’Neill, L., Rauner, M., Heidenberger, K., Kraus, M., 2008. A cross-national comparison and taxonomy of DEA-based hospital efficiency studies. SocioEconomic Planning Sciences 42 (3), 158–189. Prieto, A.M., Zofio, J.L., 2007. Network DEA efficiency in input–output models: with an application to OECD countries. European Journal of Operational Research 178, 292–304. Reiff, B., Gregor, B., 2005. Transportation Planning Performance Measures. Oregon Department of Transportation Federal Highway Administration. Scheel, H., 2001. Undesirable outputs in efficiency valuations. European Journal of Operational Research 132, 400–410. Sheth, C., Triantis, K., Teodorovic´, D., 2007. Performance evaluation of bus routes: a provider and passenger perspective. Transportation Research Part E 43, 453–478. Tone, K., 2001. A slacks-based measure of efficiency in data envelopment analysis. European Journal of Operational Research 130 (3), 498–509. Tone, K., Tsutsui, M., 2009. Network DEA: a slacks based measure approach. European Journal of Operational Research 197 (1), 243–252. Tsolakis, D., Thoresen, T., 1998. A Framework for demonstrating that road performance meets community expectation. Road and Transportation Research 7 (3), 79–85. TTI, 2005. The Keys to Estimating Mobility in Urban Areas: Applying Definition and Measures That Everyone Understands. Texas Transportation Institute. Yu, M.-M., Lin, E.T.J., 2008. Efficiency and effectiveness in railway performance using a multi-activity network DEA model. Omega 36, 1005–1017. Zhao, Y., Triantis, K., 2009. Transportation network performance measurement-the impact of a downtown space reservation system: a network-DEA approach. In: Proceedings of the 2009 Industrial Engineering Research Conference, pp. 350–355. Zhao, Y., Triantis, K., Edara, P., 2010a. Evaluation of travel demand strategies: a microscopic traffic simulation approach. Transportation 37 (3), 549–571. Zhao, Y., Triantis, K., Teodorovic´, D., Edara, P., 2010b. A travel demand management strategy: the downtown space reservation system. European Journal of Operational Research 205 (3), 584–594. doi:10.1016/j.ejor.2010.01.026. Zofio, J.L., Prieto, A.M., 2001. Environmental efficiency and regulatory standards: the case of CO2 emissions from OECD industries. Resource and Energy Economics 23 (1), 63–83.