Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics

Transportation Research Part B xxx (2015) xxx–xxx

Contents lists available at ScienceDirect

Transportation Research Part B journal homepage: www.elsevier.com/locate/trb

21st International Symposium on Transportation and Traffic Theory

Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics Hongbo Ye a, Hai Yang a, Zhijia Tan b,⇑ a b

Department of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China School of Management, Huazhong University of Science & Technology, Luoyu Road 1037, Wuhan, PR China

a r t i c l e

i n f o

Article history: Received 8 April 2015 Received in revised form 31 July 2015 Accepted 1 August 2015 Available online xxxx Keywords: System optimum User equilibrium Marginal-cost pricing Trial-and-error procedure Day-to-day flow dynamics

a b s t r a c t This paper investigates the convergence of the trial-and-error procedure to achieve the system optimum by incorporating the day-to-day evolution of traffic flows. The path flows are assumed to follow an ‘excess travel cost dynamics’ and evolve from disequilibrium states to the equilibrium day by day. This implies that the observed link flow pattern during the trial-and-error procedure is in disequilibrium. By making certain assumptions on the flow evolution dynamics, we prove that the trial-and-error procedure is capable of learning the system optimum link tolls without requiring explicit knowledge of the demand functions and flow evolution mechanism. A methodology is developed for updating the toll charges and choosing the inter-trial periods to ensure convergence of the iterative approach towards the system optimum. Numerical examples are given in support of the theoretical findings. Ó 2015 Elsevier Ltd. All rights reserved.

1. Introduction As rational road users selfishly minimize their own travel time, the user equilibrium (UE) flow patterns will usually deviate from the system optimum (SO) that is the state when the total travel time in a traffic network is at a minimum. To achieve the SO, researchers have designed various mechanisms including the so-called first-best road pricing scheme which is most well-known. Road pricing is increasingly believed to be an effective and efficient instrument to relieve traffic congestion, reduce vehicular emissions, manage travel demand and achieve transportation sustainability. The idea of road pricing can be traced back to Pigou (1920) and the followers such as Walters (1961), Beckmann (1965) and Vickrey (1969). In the context of a congested network, various mathematical models and algorithms have been proposed to determine the SO link tolls. Yang and Huang (2005), Tsekeris and Voss (2009) and de Palma and Lindsey (2011) all offer comprehensive reviews. In sharp contrast with its booming development in academia, congestion pricing has only been implemented in a dozen or so cities. Besides political reasons, there are other barriers impeding the promotion of congestion pricing from a purely economic concept to a comprehensive and practical traffic regulation policy. Exact calculation of the first-best tolls requires explicit and analytical demand functions, which are difficult to establish in practice (Walters, 1961), and the commonly used linear or exponential demand functions are usually too arbitrary and unconvincing (Li, 2002). Vickrey (1993) and Downs (1993) proposed circumventing this issue by implementing congestion pricing on a trial-and-error basis without knowing the demand functions. This idea was realized for the first time when Li (1999, 2002) gave an iterative bisection algorithm that can be applied to a homogeneous traffic stream along a single ⇑ Corresponding author. E-mail address: [email protected] (Z. Tan). http://dx.doi.org/10.1016/j.trb.2015.08.001 0191-2615/Ó 2015 Elsevier Ltd. All rights reserved.

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

2

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

expressway. The trial-and-error method allows a traffic planner to estimate or revise the tolls easily by using readily available traffic count data while requiring the travel time functions only. In the same spirit, Yang et al. (2004) suggested an algorithm based on the method of successive averages (MSA) (Powell and Sheffi, 1982) and presented a rigorous theoretical proof of its convergence in a general network. The method was later enhanced by Han and Yang (2009) with a faster convergence. Yang et al. (2005) developed a sequential bi-level programming approach for iteratively estimating traffic demand information (demand matrix or demand functions) and optimizing link tolls to deal with the second-best road pricing problem with unknown demand functions. Meng et al. (2005) and Yang et al. (2010) employed the trial-anderror method in the traffic-restrained road pricing problems. Wang and Yang (2012) and Wang et al. (2014) fixed a nonconvergence issue of the bisection method in Li (2002) and further adapted it to implement the tradable travel credit schemes for network mobility management. Xu et al. (2013) developed a trial-and-error pricing scheme for a network with multiple interacting types of vehicles and multiple time periods with interdependent demands. Zhou et al. (2015) proposed another trial-and-error congestion pricing scheme for achieving capacity restraint and SO. The trial-and-error method obviates the requirement for analytical demand functions and has proved to be efficient and promising. A critical underlying assumption in most of the abovementioned trial-and-error methods is that a user equilibrium exists and emerges spontaneously for any given toll charge (Yang et al., 2004). This assumption is idealistic and to some degree too restrictive in practice. The incontestable fact is that traffic flow on a certain road or path changes from day to day. Once a new pricing scheme is imposed or an existing one is altered, travelers will need time to learn and adjust their trip-making decisions. More realistically, even if the road users can be quickly informed of altered toll charges and a new equilibrium may be reached, the network flows could still temporarily evolve towards a new stable state in response to the adjustment of road pricing schemes. As a result, the link flows observed by the planner may not be in equilibrium at any arbitrary time. In this situation, even if the trial-and-error procedure can still be adopted, its convergence should be reexamined. Thus there is a great need for the development of efficient road pricing methods in networks taking into account day-to-day flow dynamics. Yang and Zhang (2009) formulated a type of fixed-demand day-to-day dynamics as a ‘rational behavior adjustment process’, which comprises several existing models. Similar models with elastic demand are the ‘excess payoff dynamics’ in Sandholm (2002, 2005) and the ‘excess travel cost dynamics’ (ETCD) in Li et al. (2012). Under the ETCD, travelers’ aggregate behavior will reduce the gap between the total cost and the total benefit. Recent development on the day-to-day flow dynamics includes Cantarella (2013), Guo et al. (2013, 2015), Parry and Hazelton (2013), Smith et al. (2014), Wu et al. (2013), Ye and Yang (2013). Watling and Cantarella (2013) provided a state-of-the-art review. Yang and Szeto (2006) adopted a dynamic toll scheme in the network with the ‘rational behavior adjustment process’ to achieve SO by charging the marginal-cost tolls (Button, 1993) based on the instantaneous link flows, and then extended by Yang (2007) to the elastic-demand case. Yang et al. (2007) suggested that imposing the tolls corresponding to the steepest descent direction of the total system cost could accelerate the system’s convergence to SO. Sandholm (2002) recommended a dynamic pricing mechanism to achieve the SO tolls without knowing the exact demand information in the network with the excess payoff dynamics. Guo et al. (2015) proposed a price-based congestion control scheme in a day-to-day link flow dynamical system for achieving a predetermined link flow pattern. Yet, all of these dynamic pricing schemes either require that the tolls be adjustable (continuously or daily) in response to the change in network flows or that the explicit mechanisms of the network flow evolution be (at least partially) known to the social planner. These requirements are impractical. A practical pricing mechanism would be, e.g., a piecewise-constant toll scheme adjusted not very frequently in calendar time and could be implemented on an unknown and dynamical environment. For example, Guo (2013) proposed a toll strategy to achieve the Wardrop’s UE in a network with boundedly-rational-UE-based day-to-day dynamics; Farokhi and Johansson (2015) considered the piecewise-constant marginal-cost toll of equal-length implementation period in repeated routing games with fixed demand. In this paper, we incorporate the day-to-day flow evolution and dynamics into the trial-and-error procedure for implementing the marginal-cost pricing in a general network with unknown demand functions and an unknown flow evolution mechanism. The toll level is changed at some time point, called a ‘trial’ moment, and kept constant until the next trial. During each inter-trial period, the path flows evolve following some dynamical process given some toll level. The resulting toll pattern would be a piecewise-constant dynamic pricing mechanism. An appropriate toll updating strategy and time periods between two adjacent trials are sought. The efficiency of the trial-and-error pricing scheme is important if the planning time for the whole toll-adjustment process is limited. Therefore, given either the total time horizon or the target error bound, the planner may select the combination of inter-trial periods and total number of trials that minimizes the time horizon or the error. The rest of this paper is organized as follows. Section 2 briefly reviews the trial-and-error procedure in Yang et al. (2004) and some other relevant concepts. In Section 3, the ‘excess travel cost dynamics’ is used to describe the day-to-day flow dynamics. An adaptive toll charging and updating scheme is proposed and the convergence of the whole procedure under this flow dynamics is discussed. Numerical examples are presented in Section 4 and conclusions are drawn in Section 5. 2. Overview of the trial-and-error method without flow dynamics Consider a general network consisting of a set of directed links denoted by A. Let W and Rw denote the set of origin– destination (OD) pairs and the set of paths between each OD pair w 2 W, respectively. Each link a 2 A is associated with a Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

3

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

link travel time function t a ðv a Þ, which is strictly increasing, convex and twice continuously differentiable w.r.t. its own flow v a . Each OD pair w 2 W is associated with a demand function dw ¼ Dw ðC w Þ, where Dw ðC w Þ is invertible, strictly decreasing and differentiable w.r.t. the generalized travel cost C w for that OD pair. 2.1. User equilibrium, social optimum and marginal-cost pricing ~ a ; a 2 AÞ can be obtained from the following optimization problem: ~ ¼ ðv The SO link flows v

min L1 ðv ; dÞ ¼ v;d

subject to

va ¼

XX

X

XZ

a2A

w2W

v a t a ðv a Þ 

dar f rw ;

0

dw

D1 w ðxÞdx

ð1Þ

8a 2 A;

ð2Þ

w2W r2Rw

dw ¼

X

f rw ;

8w 2 W;

ð3Þ

r2Rw

f rw P 0;

8r 2 R w ;

w 2 W;

ð4Þ

where v ¼ ðv a ; a 2 AÞ and d ¼ ðdw ; w 2 W Þ represent the link flow and demand vectors, respectively; f rw is the flow on path r 2 Rw of OD pair w 2 W; dar equals 1 if path r uses link a and 0 otherwise. On the other hand, for any given link toll pattern s ¼ ðsa ; a 2 AÞ, the user equilibrium link flows and OD demands can be acquired by solving the minimization problem

min L2 ðv ; d; sÞ ¼ v;d

X Z va a2A

½ta ðxÞ þ sa dx 

0

XZ w2W

0

dw

D1 w ðxÞdx

ð5Þ 

subject to (2)–(4), or the following variational inequality (VI) problem (Yang and Huang, 2005): finding   v satisfying conditions (2)–(4), feasible d

Þ þ s tðv  D1 d

!T 

v  v   dd

v

 d

 such that for all

ð6Þ

P 0:

Since D1 w ðÞ; w 2 W, is strictly decreasing and t a ðÞ; a 2 A, is strictly increasing, the UE link flow and corresponding OD demand pattern is unique. With minor abuse of notation, given path flow vector f, we also write the objective function of (5) as

L2 ðf; sÞ ¼ L2 ðv ; d; sÞ;

ð7Þ

where path flow f and link flow v satisfy relationship (2), and demand d and path flow f satisfy relationship (3). From the marginal-cost pricing principle, we know that the marginal-cost toll for each link is the difference between the marginal travel time and the average travel time, namely,

s~a ¼ v~ a t0a ðv~ a Þ; 8a 2 A:

ð8Þ

~a ; a 2 AÞ, can support the SO link flow pattern as an equilibrium, which can be ~ ¼ ðs It is clear that the marginal-cost tolls, s ~. readily obtained by solving the problem (2)–(5) with s ¼ s 2.2. Learning the marginal-cost toll pattern via the trial-and-error procedure When the demand functions are unknown, the SO link flows cannot be obtained from the optimization problem described in the previous subsection. Under this circumstance, the trial-and-error procedure proposed by Yang et al. (2004) can be adopted to push the system to the SO state. In their procedure, the toll for each link is revised based on a combination of the observed link flow pattern and the trial link flow pattern. We now describe the trial-and-error procedure in a general network proposed by Yang et al. (2004) for later reference. The trial-and-error procedure (Yang et al., 2004):  ð0Þ the initial UE flow pattern, and let Step 1. Set k ¼ 1. Denote by v Step 2. Estimate the toll sðkÞ by





saðkÞ ¼ v aðkÞ t0a v aðkÞ ; a 2 A:

v ð1Þ ¼ v ð0Þ be the initial trial link flows. ð9Þ

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

4

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

 ðkÞ . Step 3. Impose toll sðkÞ and observe the realized UE link flows v  ðkÞ   ðkÞ  ðkÞ      v  Step 4. Stop if v = v < g; otherwise, set k ¼ k þ 1and go to Step 5. Step 5. Revise the trial link flows

v ðkÞ by



vðkÞ ¼ v ðk1Þ þ hk v ðk1Þ  v ðk1Þ



;

ð10Þ

where hk satisfies

0 < hk 6 1;

þ1 X hk ¼ þ1;

þ1 X h2k < þ1;

k¼1

k¼1

ð11Þ

and go to Step 2.

P 2 Note that from the third condition þ1 k¼1 hk < þ1 in (11), we can derive that limk!þ1 hk ¼ 0. Two assumptions are made in Yang et al. (2004) to guarantee the convergence of the above trial-and-error method and are also required in the current study.   tðv Þ Assumption 1. is strongly monotone, i.e., there exists a positive number q such that for any feasible and 1 D ðdÞ distinct ðv 1 ; d1 Þ and ðv 2 ; d2 Þ, we have

tðv 1 Þ  tðv 2 Þ   D1 ðd1 Þ  D1 ðd2 Þ

!T 

v1  v2  d1  d2

   v 1  v 2 2  P q d d  ; 1 2

ð12Þ

  where tðv Þ ¼ ðt a ðv a Þ; a 2 AÞ and D1 ðdÞ ¼ D1 w ðdw Þ; w 2 W . Assumption 2. Assume the demand functions are bounded from above for all OD pairs. Therefore all the feasible link flows,   tðv Þ is bounded from above path flows and tolls are also bounded from above. Further, assume the Hessian matrix of 1 D ðdÞ as well. Denote a unified upper bound of all the terms by B.

Remark. Mathematically speaking, the trial-and-error procedure described above is just another alternative for solving the mathematical programming problem (5) or the VI problem (6). However, considering the context of our problem, the existing solution algorithms are not applicable. We assume that the demand functions are not available to the social planner, as well as the values of modular q and upper bound B in the above assumptions. However, at least part of this information is requisite for solving the VI problems (Facchinei and Pang, 2003), for both updating the solution and ensuring the convergence of the algorithms. As a result, the rules for iterating v ðkÞ in those solution algorithms do not work in our case.   v ðkÞ  vðkÞ is a descent direction of the optimization problem (1)–(4) at As proved in Yang et al. (2004), the vector  ðkÞ ðkÞ d d   v ðkÞ . According to Powell and Sheffi (1982) and Bazaraa et al. (2006), both v ðkÞ and v ðkÞ will converge to v~ as k ! 1, ðkÞ d ~ as well. and the tolls will approach s Note that a critical assumption adopted in the abovementioned trial-and-error procedure (in Step 3, to be precise) is that  ðkÞ reacting to each pricing trial sðkÞ is a UE link flow pattern. However, with flow the realized and observed link flow pattern v ! _ dynamics, the realized flow and demand pattern _

v_ðkÞ  v ðkÞ

!

v_ðkÞ dðkÞ

may not be the exact UE but a non-equilibrium (or an approximate

is not necessarily the descent direction of the optimization problem (1)–(4) at ðkÞ dðkÞ  d  v ðkÞ . Therefore the necessary conditions for the convergence of MSA in Powell and Sheffi (1982) may not be satisfied ðkÞ d and the convergence of the trial-and-error procedure should be reinvestigated, which will be done in the next section. equilibrium) and the vector



3. Trial-and-error procedure with flow dynamics 3.1. Excess travel cost dynamics with elastic demand Define the excess travel cost ETC rw ðfÞ for path r 2 Rw between OD pair w 2 W as the difference between the generalized path travel cost C rw ðfÞ and the travel benefit of that OD pair, D1 w ðdw Þ, i.e.,

ETC rw ðfÞ ¼ C rw ðfÞ  D1 w ðdw Þ;

8r 2 R w ;

w 2 W:

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

5

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

Definition 1 (Li et al., 2012). The path flow adjustment process

f_ ¼ FðfÞ

ð13Þ

is the excess travel cost dynamics (ETCD) if the following three conditions ðiÞ(iii) are satisfied, where FðfÞ ¼ ðF rw ðfÞ; r 2 Rw ; w 2 WÞT . (i) Dynamical system (13) admits a unique solution trajectory for any given initial condition, and the trajectory is Lipschitz continuous; (ii) ETCðfÞ  FðfÞ < 0 whenever FðfÞ – 0, where ETCðfÞ ¼ ðETC rw ðfÞ; r 2 Rw ; w 2 W Þ; (iii) f is a UE path flow pattern if FðfÞ ¼ 0. It is easy to verify that the ETCD is compatible with the rational behavior adjustment process with fixed demand (Yang and Zhang, 2009), as well as including the network tatonnement process (Friesz et al., 1994) and the projected dynamical system (Zhang and Nagurney, 1996; Nagurney and Zhang, 1997) with elastic demand as its special cases. The formulations of the network tatonnement process and projected dynamical system with elastic demand are given as

f_ rw ¼ aðmax f0; f rw  bETC rw ðfÞg  f rw Þ; and

f_ rw ¼



bETC rw ðfÞ

f rw > 0

max fbETC rw ðfÞ; 0g f rw ¼ 0

;

a > 0; b > 0; 8r 2 Rw ; w 2 W;

b > 0;

8r 2 Rw ;

ð14Þ

w 2 W:

ð15Þ

Without difficulty, the stability of the ETCD can be proved by adopting LaSalle’s invariant set theorem (Khalil, 2002; Li et al., 2012). The path flow dynamics with any initial state converges to the unique UE link flow and OD demand pattern. Furthermore, given any toll pattern s, the objective function in (5) satisfies L_ 2 ðf; sÞ ¼ ETCðfÞ  f_ < 0. Therefore L2 ðf; sÞ is

strictly decreasing with time. Note that L2 ðf; sÞ can be viewed as the potential energy of the transportation network with elastic demand (Sandholm, 2002; Peeta and Yang, 2003; Xiao et al., 2015), so condition (ii) implies that the potential energy of the network should be strictly decreasing along the evolution trajectory. 3.2. Convergence of the trial-and-error procedure with flow dynamics

We now investigate the convergence of the trial-and-error procedure described in Section 2.2. The main difference between the procedures with and without flow dynamics is that the observed link flow pattern in the former case is not UE. Suppose the pricing experiment is implemented and adjusted on a periodic basis (e.g. monthly or quarterly), and during each inter-trial period, the evolution of the network flows follows the ETCD described above. Under this circumstance, at the end of each inter-trial period, the social planner can only observe the non-equilibrium link flows and calculate the new tolls based on this observation. That is to say, after imposing toll sðkÞ , the social planner will wait for time Dk ; k ¼ 1; 2; 3; . . ., _

observe the flow v ðkÞ and adjust the tolls based on it. Therefore, the trial-and-error procedure with flow dynamics is the same _

as that without flow dynamics described in Section 2.2, with the non-equilibrium link flow pattern v ðkÞ replacing the UE link  ðkÞ for all k P 0, at each trial and at the very beginning of the trial-and-error procedure. flow pattern v _

_

Denote by dðkÞ and f ðkÞ the realized (but not observable) demand and path flow patterns under toll sðkÞ at trial k.  ðkÞ ; f ðkÞ and v  ðkÞ be the equilibrium demand, path flow and link flow patterns. During the inter-trial period Correspondingly, let d _

_

_

Dk , the path flows will evolve from f ðk1Þ to f ðkÞ following path flow dynamics (13), where f ð0Þ is the realized path flow at the beginning of the first trial. Here we intend to establish the conditions under which the symbiotic process of flow dynamics eventually converges to the desired and unknown SO flow pattern, allowing the SO toll pattern to be effectively estimated. Intuitively, under a convergent flow evolution process (such as the aforementioned ETCD which converges to UE as time tends to infinity), if each inter-trial period Dk ; k ¼ 1; 2; 3; . . ., is long enough, then the non-UE flow and demand pattern ! _

v_ðkÞ

is sufficiently close to UE and forms the descent direction of the optimization problem (1)–(4). This is summarized dðkÞ in the following proposition. Proposition 1. Under Assumptions 1 and 2, there always exist inter-trial periods fDk gþ1 k¼1 such that the link flow and toll vectors, n_ oþ1

ðkÞ þ1 ðkÞ ~ ~. v and s , converge to the SO link flow pattern v and the SO toll vector s k¼1 k¼1

Proof. Firstly, we know that, given sðkÞ ;



v ðkÞ  ðkÞ d



_

is fixed, and limDk !1

v_ðkÞ dðkÞ

!

 ¼

  ðkÞ . Then referring to Proposition 2 in d

v ðkÞ

Yang et al. (2004), Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

6

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

lim rL1

Dk !1



v

ðkÞ

;d

ðkÞ

T

_

v ðkÞ  v ðkÞ _ ðkÞ

d

d

! ¼ rL1

ðkÞ



v

ðkÞ

ðkÞ

;d

T

v ðkÞ  v ðkÞ  ðkÞ  dðkÞ d

!

2   v   ðkÞ  v ðkÞ  6 q  ðkÞ  :  d  dðkÞ 

ð16Þ

Then there must exist Dk > 0 such that

rL1



v ðkÞ ; dðkÞ

T

_

v ðkÞ  vðkÞ _

!

ðkÞ

dðkÞ  d

_

which implies that the vector

v_ðkÞ dðkÞ

2  q  v ðkÞ  v ðkÞ  6    ðkÞ  ; 2  d  dðkÞ  !

 

vðkÞ



ðkÞ

d

ð17Þ

is a feasible descent direction of the objective function L1 ðv ; dÞ at

Since the SO problem is strictly convex and has a unique minimum, we have k ! 1. h

v

ðkÞ

~ , and thus s !v

ðkÞ

_

~; v !s

ðkÞ



v ðkÞ ðkÞ

d

 .

~ as !v

Proposition 1 indicates that, once an appropriate sequence of the inter-trial periods are selected, the convergence of the dynamic link flow pattern to SO is guaranteed. However, selecting the appropriate inter-trial periods fDk gþ1 k¼1 is still difficult since the exact UE link flow pattern at each trial is unknown. Choosing an inappropriate Dk (which is likely since the flow dynamics is not clear to the planner) will violate condition (17) and the proof above will be invalidated. Thus the length of the inter-trial periods seems to play a key role in shaping the convergence of the trial-and-error procedure. Technically speaking, a long inter-trial period would ensure that the actual flow is ‘close enough’ to an equilibrium and thus the tolls could be adjusted more appropriately in each trial. A short inter-trial period, on the other hand, would allow the tolls to be adjusted more frequently based on a still evolving disequilibrium flow pattern. Hereinafter, we will show that, even when the inter-trial periods are not long enough, the trial-and-error procedure can still converge to SO. For simplicity, we only consider identical inter-trial periods, namely, Dk ¼ D > 0 for all k P 1. This however is not necessary for the convergence of the procedure. Setting adaptive or non-uniform inter-trial periods may require more information on the demand functions and/or flow dynamics. We leave this topic for future research. To ensure the convergence of the trial-and-error procedure, we first introduce the following assumption on the day-today flow dynamics. Assumption 3. Given any feasible initial path flow and link toll pattern, the relative decreasing rate along the path flow trajectory determined by dynamical system (13) is not greater than a decreasing function dðsÞ with dð0Þ ¼ 1 and lims!þ1 dðsÞ ¼ 0, i.e.,

  L2 ðfðsÞ; sÞ  L2 f; s   6 dðsÞ; L2 ðf 0 ; sÞ  L2 f; s

ð18Þ

where f 0 is an initial path flow pattern and f is the resultant UE path flow pattern under link toll pattern s. Characteristics of the unknown flow dynamics essentially affect the performance of the trial-and-error procedure. Assumption 3 or inequality (18) is a stronger assumption on the day-to-day dynamics, which requires that the adjustment process of the path flows to UE be sufficiently ‘fast’. Note that dðsÞ is independent of the initial path flows and toll charges, and thus travelers are expected to be ‘smart enough’ to learn their optimum. It must be pointed out that, the dynamical systems such as the network tatonnement process (14) or the projected dynamical system (15)would not necessarily satisfies condition (18) when the initial path flow pattern f 0 is very close to UE f. And it is not easy either to find or to verify a day-today process satisfying Assumption 3. For an exponentially stable dynamical system with a certain initial link flow and OD demand pattern, dðsÞ can be estimated from the Lipschitz constant of the link travel time functions, inverse demand functions and the exponential power. The rest of this section will show that, based on Assumption 3, we can relax the requirement on the length of the intertrial periods in Proposition 1, while maintaining the effectiveness of the trial-and-error scheme. Proposition 2 shows that, with identical inter-trial periods, the realized non-UE flows will eventually approach UE. However, under this circumstance, the results in Section 2.2 or Proposition 1 cannot be directly applied. Thus in Proposition 3, we further prove the convergence of our trial-and-error scheme to SO. _  _   ðkÞ    ðkÞ  both approach zero when k ! 1.  ðkÞ  Proposition 2. Under Assumptions 1–3, the Euclidean distances v ðkÞ  v d  d  and    Proof. To begin with, from (6), we have

!T  ðk1Þ   t v þ sðk1Þ   ðk1Þ  D1 d

v ðkÞ  v ðk1Þ  ðkÞ  d  ðk1Þ d

! P0

ð19Þ

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

7

and

!T  ðkÞ   t v þ sðkÞ   ðkÞ  D1 d

v ðk1Þ  v ðkÞ

! ð20Þ

P 0:

 ðk1Þ  d  ðkÞ d

Adding up inequalities (19) and (20) gives rise to

 ðk1Þ  !  t v    1  ðk1Þ D d

"

 ðkÞ  !  t v   þ 1  ðkÞ D d

sðk1Þ  sðkÞ

!#T

!

v ðkÞ  v ðk1Þ

P0

 ðkÞ  d  ðk1Þ d

0

ð21Þ

or equivalently

sðk1Þ  sðkÞ

!T

v ðkÞ  v ðk1Þ

"

! P

 ðkÞ  d  ðk1Þ d

0

2  !  ðk1Þ  !# v  t v  ðk1Þ  v ðkÞ  v ðk1Þ    ðkÞ  v ;   ðk1Þ   ðkÞ  d  ðkÞ  d  ðk1Þ P q  ðk1Þ   d d D1 d

 ðkÞ  !  t v   ðkÞ   D1 d

ð22Þ

where the second inequality comes from Assumption 1. Therefore we have

   ðk1Þ   ðkÞ  v  ðk1Þ   ðkÞ  v s  s   ðkÞ  ðk1Þ  P  d d

!T

sðk1Þ  sðkÞ

v ðkÞ  v ðk1Þ

!

 ðkÞ  d  ðk1Þ d

0

2  v  ðk1Þ     ðkÞ  v P q  ðkÞ  ðk1Þ  ;  d d

ð23Þ

and thus

  v   ðk1Þ   1   ðkÞ  v   ðkÞ  ðk1Þ  6 sðk1Þ  sðkÞ :  q d d

ð24Þ

Since the link travel time functions t a ðv a Þ; a 2 A, are twice differentiable, then the marginal-cost toll function  

sðv Þ ¼ sa ¼ v a t0a ðv a Þ; a 2 A T , given by (8), is continuously differentiable and thus Lipschitz continuous. Therefore, there

exists a Lipschitz constant M 1 > 0 such that the right-hand side of (24) can be bounded from above by

_   ðk1Þ      s  sðkÞ  6 M 1 v ðkÞ  v ðk1Þ  ¼ M 1 hk v ðk1Þ  v ðk1Þ  6 M1 Bhk ;

ð25Þ

where the equality follows Eq. (10) and the last inequality follows Assumption 2. Now, according to Eq. (18), we have

_   _

    L2 f ðkÞ ; sðkÞ  L2 f ðkÞ ; sðkÞ 6 dðDÞ L2 f ðk1Þ ; sðkÞ  L2 f ðkÞ ; sðkÞ  _  _       L2 f ðk1Þ ; sðkÞ  L2 f ðk1Þ ; sðk1Þ ¼ dðDÞ L2 f ðk1Þ ; sðkÞ  L2 f ðk1Þ ; sðk1Þ  _

        þ L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ þ L2 f ðk1Þ ; sðkÞ  L2 f ðkÞ ; sðkÞ ¼ dðDÞ

( _ðk1Þ X Z va  v ðk1Þ a

a2A

þ

" ðk1Þ X Z v a v ðkÞ a

a2A

¼ dðDÞ þ



ta ðxÞdx 

a2A

v ðkÞ a

XZ w2W

sðkÞ  sðk1Þ

" ðk1Þ X Z v a

_



   saðkÞ  saðk1Þ dx þ L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ

T _ðk1Þ

v

ta ðxÞdx 

ðk1Þ d w

ðkÞ d w

# D1 w ð

xÞdx þ

ðk1Þ X Z v a

a2A

v ðkÞ a

)

s

ðkÞ a d

x



 _    ðk1Þ þ L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ v

XZ w2W

ðk1Þ d w

ðkÞ d w

#



  ðkÞ T  ðk1Þ

xÞdx þ s

D1 w ð

v

 ðkÞ v



)



 _      _  ðk1Þ  6 dðDÞ  sðkÞ  sðk1Þ   v ðk1Þ  v  þ L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ " # ) X  X  ðk1Þ   ðkÞ   ðk1Þ  ðk1Þ ðkÞ ðkÞ ðkÞ   v d   ;   v  þ  d  þ s   v v þB a

a2A

a

w

w

ð26Þ

w2W

where the last inequality follows Assumption 2. Referring to Eq. (24), we know that

 pffiffiffiffi  X  X  ðk1Þ ðkÞ  pffiffiffiffi  ðk1Þ  v  ðkÞ  l  ðk1Þ ðkÞ   v v ðk1Þ  v ðkÞ  þ d   dw 6 l  ðk1Þ  ðkÞ  6 s ; s a a w   q d d w2W a2A

ð27Þ

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

8

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

  v ðkÞ where l is the dimension of vector  ðkÞ . Referring to Assumption 2, for the right-hand side of Eq. (26), we have d _   ðkÞ   ðk1Þ  ðk1Þ    v 6 B. Further substituting Eq. (27) into Eq. (26) yields v  6 B and s

_   _

    L2 f ðkÞ ; sðkÞ  L2 f ðkÞ ; sðkÞ 6 dðDÞ  L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ  pffiffiffiffi    l  ðk1Þ ðkÞ    ðk1Þ  ðkÞ  s v s þB v þ dðDÞ  BsðkÞ  sðk1Þ  þ B q  _



    1 pffiffiffiffi 6 dðDÞ  L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ þ dðDÞ  B 1 þ ð l þ 1Þ sðkÞ  sðk1Þ  q  _



  1 pffiffiffiffi 6 dðDÞ  L2 f ðk1Þ ; sðk1Þ  L2 f ðk1Þ ; sðk1Þ þ dðDÞ  M 1 B2 1 þ ð l þ 1Þ hk ; ð28Þ

q

where the second inequality is derived from Eq. (24), and the third inequality holds due to Eq. (25). For simplicity and without confusion, dðDÞ is written as d. From Eq. (28), we further have

_  

X _

k     1 pffiffiffiffi L2 f ðkÞ ; sðkÞ  L2 f ðkÞ ; sðkÞ 6 dk1 L2 f ð1Þ ; sð1Þ  L2 f ð1Þ ; sð1Þ þ M 1 B2 1 þ ð l þ 1Þ hi dkiþ1 :

q

ð29Þ

i¼2

For the first term on the right-hand side of Eq. (29), we know that

 _

   lim dk1  L2 f ð1Þ ; sð1Þ  L2 f ð1Þ ; sð1Þ ¼ 0:

ð30Þ

k!1

We now consider the second term on the right-hand side of Eq. (29). If 0 < d < 1, then dk is strictly increasing with k and P limk!1 dk ¼ þ1. Denote bk ¼ ki¼2 hi diþ1 . If limk!1 hk ¼ 0, then

bk  bk1 lim k!1 dk  dðk1Þ

¼ lim

hk dkþ1

k!1 dk ð1

 dÞ

¼ lim

hk d ¼ 0: d

k!1 1

By the Stolz–Cesàro Theorem (Theorem 1.22, Muresan, 2009),

( ) ( ) k k X X bk bk  bk1 kiþ1 k iþ1 ¼ lim d ¼ lim k ¼ lim k lim hi d hi d ¼ 0: k!1 k!1 k!1 d k!1 d  dðk1Þ i¼2 i¼2

_   ðkÞ  f ðkÞ  ¼ 0 for all k P 1 and all D > 0, and the problem degenerates to the case without flow dynamics and If d ¼ 0, then  f   

P ¼ 0. Therefore when 0 6 dðDÞ < 1 with hk satisfying conditions (11), we have limk!1 ki¼2 hi dkiþ1 ¼ 0. Together _    with Eqs. (29) and (30), and also considering that L2 f ðkÞ ; sðkÞ P L2 f ðkÞ ; sðkÞ , then by the squeeze theorem, Pk

kiþ1 i¼2 hi d

 _

  lim L2 f ðkÞ ; sðkÞ  L2 f ðkÞ ; sðkÞ ¼ 0

ð31Þ

k!1

or equivalently

 

_  ðkÞ  ðkÞ ðkÞ  _  ;d ;s lim L2 v ðkÞ ; dðkÞ ; sðkÞ  L2 v ¼ 0:

ð32Þ

k!1

Since



v ðkÞ ; d ðkÞ



is the unique minimum of L2

_   ðkÞ   ðkÞ  limk!1  d  d  ¼ 0. h



v; d; sðkÞ

_     ðkÞ  , Eq. (32) indicates that limk!1 v ðkÞ  v  ¼ 0 and

Proposition 2 states that, under certain assumptions, the gap between the observed non-equilibrium link flow and OD demand pattern and the corresponding UE pattern approaches zero as the trial-and-error procedure runs. No matter how short the uniform inter-trial period D is, as long as the day-to-day dynamics satisfies Assumption 3, the observed flow will eventually reach UE. However, this does not necessarily mean that the convergence results in Yang et al. (2004) can be directly applied here, since the realized flow and demand pattern cannot always assure a descent direction for the objective function L1 ðv ; dÞ in Eq. (1). The following result concludes that the observed link flow pattern does converge to SO. The basic

þ1 idea is to split the sequence v ðkÞ k¼1 into two subsequences based on whether condition (34) is satisfied or not. By showing that both subsequences converge to SO, we prove the convergence of the trial-and-error procedure.

þ1 ~. Proposition 3. Under Assumptions 1–3, the toll pattern sðkÞ k¼1 converges to the SO link toll pattern s

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

9

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

_  _   ðkÞ    ðkÞ  ¼ 0. Then there exists a  ðkÞ  Proof. By Proposition 2, under Assumptions 1–3, limk!1 v ðkÞ  v d  d  ¼ 0 and limk!1    _  _   ðkÞ   ðkÞ  ðkÞ  þ1  ðkÞ  non-negative sequence fek gk¼1 such that limk!1 ek ¼ 0; v  v  6 ek and  d  d  6 ek for all k P 1. According to Assumption 2 and Eq. (16), we have

rL1



v ðkÞ ; dðkÞ

T

_

v ðkÞ  v ðkÞ _

dðkÞ  d

!

¼ rL1

ðkÞ



   6 rL1 v ðkÞ ; d

T

!

!

 T v  ðkÞ  v ðkÞ v ðkÞ  v ðkÞ ðkÞ þ rL1 v ðkÞ ; d _ ðkÞ  dðkÞ  d  ðkÞ dðkÞ  v  2 2 _     v v   ðkÞ  v  ðkÞ     ðkÞ  v ðkÞ    ðkÞ  v ðkÞ  ðkÞ   v

v ðkÞ ; dðkÞ

_

ð33Þ

  q  ðkÞ  6 Bek  q  ðkÞ  :   _  dðkÞ  d  d  dðkÞ   d  dðkÞ   ðkÞ 

If the following inequality always holds,

2 2    v q  v ðkÞ  vðkÞ    ðkÞ  v ðkÞ  Bek  q  ðkÞ 6     ðkÞ ðkÞ  ;  d  dðkÞ  2d d 

ð34Þ

or equivalently

2   v 2Bek   ðkÞ  v ðkÞ  ;   ðkÞ ðkÞ  P d d  q _

then

v_ðkÞ  v ðkÞ d

ðkÞ

ð35Þ

! is the descent direction of L1

ðkÞ

d



vðkÞ ; dðkÞ



. Similar to the proof of Proposition 1, we immediately know that

  v  ðkÞ  v ðkÞ   Proposition 3 holds. On the other hand, if (34) is violated for all k P 1, then limk!1   ðkÞ  dðkÞ  ¼ 0 since limk!1 ek ¼ 0. d Therefore Proposition 3 is also true. We now discuss the case where inequality (34) is neither always true nor always violated. We divide the sequence

ðkÞ þ1 v k¼1 into two subsequences based on whether condition (34) is satisfied or not. The subsequence violating condition (34) will obviously converge to SO since limk!1 ek ¼ 0. And as we will prove later, the subsequence satisfying condition (34)

þ1 will converge to SO as well. So the sequence v ðkÞ k¼1 will converge to SO. Suppose Eq. (34) is violated at trial k ¼ ki but holds for all trials l 2 fki þ 1; ki þ 2; . . . ; ki þ jg; j P 1. For trial ki ,

2   v 2Beki   ðki Þ  v ðki Þ  :   ðk Þ ðki Þ  < i d d  q

Then by Eqs. (24) and (25), we know that

sffiffiffiffiffiffiffiffiffiffiffi            ðki þ1Þ  v    ðki Þ  v ðki Þ  M B v   ðki Þ  2Beki   v ðki þ1Þ  v ðki Þ   v    ðki þ1Þ  v ðki þ1Þ   v 1 6 þ þ < : h þ Bh þ       ðk þ1Þ    ki þ1  ðki þ1Þ  d  ðki Þ  dðki Þ   ðki Þ   dðki þ1Þ  dðki Þ   d  d i  dðki þ1Þ   d q ki þ1 q

Thus

sffiffiffiffiffiffiffiffiffiffiffi      _    v ðki þ1Þ  v ðki þ1Þ   _   ðki þ1Þ  v ðki þ1Þ  ðk þ1Þ  ðki þ1Þ  2Beki M1  v i  v  v   : þ 1 Bhki þ1 þ  6 _  þ   ðk þ1Þ _ ðk þ1Þ  < eki þ1 þ i  dðki þ1Þ  dðki þ1Þ   dðki þ1Þ  d  ðki þ1Þ   d q q d i 

ð36Þ

 Assume that Eq. (34) holds for all l 2 ki þ 1; ki þ 2; . . . ; kj . Then by Taylor’s expansion and Eq. (10),

L1



v

ðlþ1Þ

ðlþ1Þ

;d



 L1



v

ðlÞ

;d

ðlÞ



¼ hlþ1 rL1



v

ðlÞ

ðlÞ

;d

T

_ ðlÞ

v

_ ðlÞ

 v ðlÞ

!

ðlÞ

d d !T ! _   _ ðlÞ ðlÞ v v v ðlÞ  v ðlÞ 1 2 ðlÞ ðlþ1Þ 2 ðlÞ ðlþ1Þ r L1 nv þ ð1  nÞv ; nd þ ð1  nÞd þ hlþ1 _ _ ðlÞ ðlÞ 2 dðlÞ  d dðlÞ  d   2  ðlÞ  _ 2 q  v  v ðlÞ  1 2  v ðlÞ  v ðlÞ  6  hlþ1   ðlÞ  þ Bhlþ1  _  ðlÞ   dðlÞ  dðlÞ  2 2 d d  2  ðlÞ !2 v   v ðlÞ  q  v ðlÞ  v ðlÞ  1 2   6  hlþ1   ðlÞ þ Bhlþ1 el þ   ðlÞ ðlÞ ðlÞ 2 2 d d  d d  " #  ðlÞ 2  ðlÞ  v v   v ðlÞ    v ðlÞ  1 2    ð37Þ ¼ hlþ1 ðBhlþ1  qÞ d  ðlÞ  dðlÞ  þ 2Bhlþ1 el  d  ðlÞ  dðlÞ  þ Bhlþ1 el ; 2

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

10

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

where n 2 ð0; 1Þ, and the first inequality follows Eqs. (33), (34) and Assumption 2. Note that limk!1 el ¼ 0 and liml!1 hlþ1 ¼ 0. Thus when l is large enough, we always have Bhlþ1  q < 0, and

sffiffiffiffiffiffiffiffiffi 2Bel

pffiffiffiffiffiffiffiffiffiffiffi

pffiffiffiffiffiffiffi el Bhlþ1  ffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffi ¼ Bel q pq  Bhlþ1

sffiffiffiffi 2

pffiffiffiffiffiffiffiffiffiffiffiffi ! el hlþ1  ffi pffiffiffiffiffiffiffiffiffiffiffi > 0; q pffiffiffi q  Bhlþ1

where the second term on the left-hand side of the equality is the larger root of the following equation

ðBhlþ1  qÞx2 þ 2Bhlþ1 el x þ Bhlþ1 e2l ¼ 0;

ð38Þ

which will guarantee that

2      v v   ðlÞ  v ðlÞ    ðlÞ  v ðlÞ  2 ðBhlþ1  qÞ  ðlÞ ðlÞ  þ 2Bhlþ1 el   ðlÞ ðlÞ  þ Bhlþ1 el < 0 d d  d d 

ð39Þ

holds for all

sffiffiffiffiffiffiffiffiffi   pffiffiffiffiffiffiffiffiffiffiffi  v 2Bel el Bhlþ1   ðlÞ  v ðlÞ  pffiffiffiffiffiffiffiffiffiffiffi : P >    ðlÞ p ffiffiffi ffi  d  dðlÞ  q q  Bhlþ1

ð40Þ

n  oki þj ðlÞ From Eqs. (37) and (39), we immediately know that L1 v ðlÞ ; d is strictly decreasing and bounded from above by l¼ki þ1 _  _    _    ðk þ1Þ ðki þ1Þ  ðki þ1Þ  ! 0 as k ! 1. Thus _  ðki þ1Þ  . By Eq. (36), v ðki þ1Þ  v d v ðki Þ ! v~ ; dðki Þ ! d~ and then L1 v ðki þ1Þ ; d i  ! 0 and  i d  L1



v ðk Þ ; dðk Þ i

i



n  oki þj   ~ . Therefore L1 v ðlÞ ; dðlÞ ~; d ! L1 v

l¼ki þ1

n_ oki þj   ðlÞ ~ and then v ~; d ! L1 v

~ , therefore we conclude that v sequence satisfying condition (34) will converge to v obtained. h

l¼ki þ1 ðkÞ

~ . Now as proved above, the sub!v

~ and the SO tolls will be will approach v

Proposition 3 shows that, with certain conditions in Assumptions 1 and 2, as long as the observed flow will gradually approach UE along the trial-and-error procedure, the link flow trajectory starting from any initial state resulting from the trial-and-error procedure will approach the SO link flow pattern. The procedure is still efficient when the evolution of path flows follows a day-to-day adjustment process and the link tolls need not be adjusted day by day. Specifically, convergence of the observed flow pattern to UE is ensured by Assumption 3, regardless of the length of the inter-trial periods. However, Assumption 3 requires that all travelers be very sensitive to their travel cost and adjust their routes quickly. The trial-anderror procedure would not be able to obtain the SO link tolls in an efficient manner if some of the travelers are inertial and do not change their routes even when the travel cost is excessive. 4. A numerical example In this section, the same example used in Yang et al. (2004) is adopted to illustrate the proposed trial-and-error scheme with day-to-day path flow dynamics. The network, as shown in Fig. 1, consists of 7 nodes, 11 links and 4 OD pairs (1 ! 7; 2 ! 7; 3 ! 7 and 6 ! 7). The true but unknown inverse demand functions are given as follows:

1 0:04 1 D1 2!7 ðd2!7 Þ ¼  0:03 1 D1 3!7 ðd3!7 Þ ¼  0:05 1 D1 6!7 ðd6!7 Þ ¼  0:05

D1 1!7 ðd1!7 Þ ¼ 

d1!7 ; 600 d2!7 ln ; 500 d3!7 ln ; 500 d6!7 ln : 400 ln

The link travel time functions follow the BPR (Bureau of Public Roads, 1964) form with the free flow travel time t 0a and link capacity ca given in Table 1.

" ta ðv a Þ ¼

t 0a

 4 # va 1 þ 0:15 ; ca

a 2 A:

ð41Þ

The day-to-day path flow dynamics is assumed to be given by Eq. (15) with b ¼ 1. Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

11

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

3 7

10

4

1

3

11

1 2

5

5

9

6 8

2

7

4

6

Fig. 1. Network structure.

Table 1 Parameters of the link travel time functions. Link no., a Free flow travel time, t 0a Link capacity, ca

1 6 200

2 5 200

3 6 200

4 7 200

5 6 100

6 1 100

7 5 150

8 10 150

9 11 200

10 11 200

11 15 200

4.1. Convergence of the link flow trajectories to the SO link flow pattern The initial flows are 50 on all paths. All inter-trial periods have the same length of 10. It is chosen that hk ¼ 1=k. Fig. 2 shows the evolution of the realized link flows, which converge to a stable flow pattern after 15 trials. _

~ is shown in Fig. 3. It can The change in the Euclidean distance between the realized link flows v ðkÞ and the SO link flows v be seen that the distance approaches zero very quickly in 15 trials.

4.2. Trial-and-error procedure with different values of hk We now examine the convergence of link flows under different sequences of hk . The initial flows are 50 on all paths. The inter-trial periods are of an identical length of 5. Fig. 4 shows the Euclidean distance between the realized flows and the SO 0:5

1

1:1

link flows with (a) hk ¼ k , (b) hk ¼ k , and (c) hk ¼ k , respectively. Sequence (b) satisfies conditions (11) while sequences (a) and (c) do not. All three sequences lead to the convergence of the realized flows toward SO. Further, in this a example, when hk ¼ k ; a > 0, a small a may correspond to a higher convergence speed.

Fig. 2. Evolution of realized link flows.

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

12

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

Fig. 3. Euclidean distance between realized link flows

_ ðkÞ

v

~. and SO link flows v

0:5

Fig. 4. Euclidean distance between the realized flows and the SO flows with (a) hk ¼ k

1

1:1

, (b) hk ¼ k , and (c) hk ¼ k

.

4.3. Trial-and-error with different time horizons and different inter-trial periods As the cost for toll-adjustment could be substantial in practice, the inter-trial periods D cannot be too short. Under this circumstance, we must make a trade-off between the adjustment cost and the convergence rate. But how should the time horizon be chosen for the trial-and-error implementation, and how should the lengths of all inter-trial periods be determined throughout the implementation? In this section, we investigate how the identical length D of the inter-trial periods and the number of total trials affect the convergence speed. The aim is to select the optimal D and total number of trials to achieve a target error while minimizing the total implementation time, or to achieve the minimal error within a predetermined finite amount of time or cost budget. As in the above examples, the error or efficiency of the trial-and-error procedure could be assessed by the Euclidean distance between the observed and SO link flows in each trial. Given a time horizon T, changing the total number of trials K will change the length of the inter-trial period according to D ¼ T=K. For each combination of time horizon T and total number of trials K, we can calculate the Euclidean distance between the realized flows and the SO flows at the end of the time horizon, as depicted in Fig. 5. In Fig. 5, each line corresponds to a certain T between 50 and 400 and various values of K. Some observations can be made from the figures. Firstly, generally speaking, given the time horizon T and hk , it may be beneficial to have shorter inter-trial periods and more trials. Secondly, in this example, no matter how long the time horizon T is, the minimum error can be a approximately achieved within 10 to 20 trials. Thirdly, when hk ¼ k , a > 0, by comparing the three figures for the same time horizon and the same number of total trials, one can see that a ¼ 0:5 always gives the minimal error and a ¼ 1:1 the maximum. Thus a smaller a leads to a smaller error. Finally, once the inter-trial periods are chosen properly, a longer time horizon will always give a smaller system error. In summary, in order to achieve the minimal error associated with 0:5

the trial-and-error procedure, a longer time horizon is always preferred; once the time horizon is determined, hk ¼ k

is

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

13

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

(a) θ k = k

− 0.5

(b) θ k = k

(c) θ k = k

−1

−1.1

Fig. 5. Euclidean distance for different time horizons and different inter-trial periods: (a) hk ¼ k

0:5

1

1:1

; (b) hk ¼ k ; (c) hk ¼ k

.

recommended, and a total of 10 to 20 trials are enough for practical implementation bearing in mind the implementation cost. 5. Conclusions This paper investigated the trial-and-error implementation of road pricing for achieving the system optimum under dayto-day flow dynamics. Neither the demand functions nor the flow evolution mechanism is needed explicitly. The day-to-day flow evolution follows the ‘excess travel cost dynamics’ and the iterative updating procedure in Yang et al. (2004) is adopted. The convergence of the trial-and-error method is guaranteed under certain conditions on the flow evolution process. The inter-trial periods can be identical with arbitrary length, and thus the scheme is not restrictive to implement in practice. A numerical example was presented and the convergence of the trial-and-error procedure was demonstrated. The performance of the procedure was examined under different MSA parameters, time horizons and lengths of inter-trial periods. It was shown that the MSA parameters affect the convergence speed as well as the system error. To reduce the system error, more trials and shorter inter-trial periods should be adopted at the expense of a higher implementation cost. Choosing a proper combination of total number of trials and length of inter-trial periods would allow the traffic planner to strike a balance between system error and implementation cost. In this paper, the pricing design problem was analyzed under realistic conditions of traffic flow evolution. The results show the robustness of the trial-and-error congestion pricing scheme which allows for unknown demand functions in a dynamic and non-equilibrium network. The trial-and-error method has the potential to assist in the design, implementation and evaluation of various urban road pricing schemes. Particularly, the traffic planner can easily estimate or revise the tolls by using readily available traffic count data. It would be interesting, yet challenging, to study the optimal combination of total number of trials and lengths of intertrial periods for a given finite time horizon, and to theoretically investigate the error bound of the trial-and-error procedure. Other research directions include designing the system optimum anonymous link toll scheme for a transportation network with heterogeneous users, and extending the trial-and-error procedure to allow bounded rationality in travelers’ route choice. Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001

14

H. Ye et al. / Transportation Research Part B xxx (2015) xxx–xxx

Acknowledgements The authors wish to express their thanks to Mike Smith and three anonymous reviewers for their useful comments on an early version of the paper. The work described in this paper was supported by the Research Grants Council of the Hong Kong Special Administrative Region, China (Project No. HKUST16211114) and the National Natural Science Foundation of China (71201067, 71471068 and 71371020). References Bazaraa, M.S., Sherali, H.D., Shetty, C.M., 2006. Nonlinear Programming: Theory and Algorithms. John Wiley, New York. Beckmann, M.J., 1965. On optimal tolls for highways, tunnels and bridges. In: Edie, L.C., Herman, R., Rothery, R. (Eds.), Vehicular Traffic Science. Elsevier, New York. Bureau of Public Roads, 1964. Traffic Assignment Manual. US Department of Commerce. Button, K.J., 1993. Transport Economics, second ed. Edward Elgar, England. Cantarella, G.E., 2013. Day-to-day dynamic models for intelligent transportation systems design and appraisal. Transportation Research Part C 29, 117–130. de Palma, A., Lindsey, R., 2011. Traffic congestion pricing methodologies and technologies. Transportation Research Part C 19 (6), 1377–1399. Downs, A., 1993. Point of view: implementing peak-hour road pricing at fullscale: finding solution to practical problems. TR News 167, 7–9. Facchinei, F., Pang, J.S., 2003. Finite-Dimensional Variational Inequalities and Complementarity Problems, vol. 1. Springer, Berlin. Farokhi, F., Johansson, K.H., 2015. A piecewise-constant congestion taxing policy for repeated routing games. Transportation Research Part B 78, 123–143. Friesz, T.L., Bernstein, D., Mehta, N.J., Tobin, R.L., Ganjalizadeh, S., 1994. Day-to-day dynamic network disequilibria and idealized traveler information systems. Operations Research 42 (6), 1120–1136. Guo, R., Yang, H., Huang, H., 2013. A discrete rational adjustment process of link flows in traffic networks. Transportation Research Part C 34, 121–137. Guo, R.Y., Yang, H., Huang, H.J., Tan, Z.J., 2015a. Link-based day-to-day network traffic dynamics and equilibria. Transportation Research Part B 71, 248–260. Guo, R.Y., Yang, H., Huang, H.J., Tan, Z.J., 2015b. Day-to-day flow dynamics and congestion control. Transportation Science, doi:http://dx.doi.org/10.1287/ trsc.2015.0596. Guo, X., 2013. Toll sequence operation to realize target flow pattern under bounded rationality. Transportation Research Part B 56, 203–216. Han, D., Yang, H., 2009. Congestion pricing in the absence of demand functions. Transportation Research Part E 45 (1), 159–171. Khalil, H.K., 2002. Nonlinear Systems, third ed. Prentice Hall, Upper Saddle River, NJ. Li, M.Z.F., 1999. Estimating congestion toll by using traffic count data – Singapore’s area licensing scheme. Transportation Research Part E 35 (1), 1–10. Li, M.Z.F., 2002. The role of speed–flow relationship in congestion pricing implementation with an application to Singapore. Transportation Research Part B 36 (8), 731–754. Li, Y., Tan, Z., Chen, Q., 2012. Dynamics of a transportation network model with homogeneous and heterogeneous users. Discrete Dynamics in Nature and Society 2012, 1–16. Meng, Q., Xu, W., Yang, H., 2005. Trial-and-error procedure for implementing a road-pricing scheme. Transportation Research Record 1923, 103–109. Muresan, M., 2009. A Concrete Approach to Classical Analysis. Springer, New York. Nagurney, A., Zhang, D., 1997. Projected dynamical systems in the formulation, stability analysis, and computation of fixed-demand traffic network equilibria. Transportation Science 31 (2), 147–158. Parry, K., Hazelton, M.L., 2013. Bayesian inference for day-to-day dynamic traffic models. Transportation Research Part B 50, 104–115. Peeta, S., Yang, T., 2003. Stability issues for dynamic traffic assignment. Automatica 39 (1), 21–34. Pigou, A.C., 1920. The Economics of Welfare. MacMillan, London. Powell, W.B., Sheffi, Y., 1982. The convergence of equilibrium algorithms with predetermined step sizes. Transportation Science 16 (1), 45–55. Sandholm, W.H., 2002. Evolutionary implementation and congestion pricing. The Review of Economic Studies 69 (3), 667–689. Sandholm, W.H., 2005. Excess payoff dynamics and other well-behaved evolutionary dynamics. Journal of Economic Theory 124 (2), 149–170. Smith, M., Hazelton, M.L., Lo, H.K., Cantarella, G.E., Watling, D.P., 2014. The long term behaviour of day-to-day traffic assignment models. Transportmetrica A 10 (7), 647–660. Tsekeris, T., Voss, S., 2009. Design and evaluation of road pricing: state-of-the-art and methodological advances. Netnomics 10 (1), 5–52. Vickrey, W.S., 1969. Congestion theory and transport investment. American Economic Review 59 (2), 251–260. Vickrey, W., 1993. Point of view: principles and applications of congestion pricing. TR News 167, 4–5. Walters, A.A., 1961. The theory and measurement of private and social cost of highway congestion. Econometrica 29 (4), 676–699. Wang, X., Yang, H., 2012. Bisection-based trial-and-error implementation of marginal cost pricing and tradable credit scheme. Transportation Research Part B 46 (9), 1085–1096. Wang, X., Yang, H., Han, D., Liu, W., 2014. Trial and error method for optimal tradable credit schemes: the network case. Journal of Advanced Transportation 48 (6), 685–700. Watling, D.P., Cantarella, G.E., 2013. Modelling sources of variation in transportation systems: theoretical foundations of day-to-day dynamic models. Transportmetrica B 1 (1), 3–32. Wu, J., Sun, H., Wang, D.Z.W., Zhong, M., Han, L., Gao, Z., 2013. Bounded-rationality based day-to-day evolution model for travel behavior analysis of urban railway network. Transportation Research Part C 31, 73–82. Xiao, F., Yang, H., Ye, H., 2015. Day-to-day network flow dynamics with inertia, working paper. Xu, W., Yang, H., Han, D., 2013. Sequential experimental approach for congestion pricing with multiple vehicle types and multiple time periods. Transportmetrica B 1 (2), 136–152. Yang, F., Szeto, W.Y., 2006. Day-to-day dynamic congestion pricing policies towards system optimal. In: Proceedings of the First International Symposium on Dynamic Traffic Assignment, Leeds, United Kingdom, pp. 266–275. Yang, F., 2007. Day-to-day dynamic optimal tolls with elastic demand, working paper. Yang, F., Yin, Y., Lu, J., 2007. Steepest descent day-to-day dynamic toll. Transportation Research Record 2039, 83–90. Yang, F., Zhang, D., 2009. Day-to-day stationary link flow pattern. Transportation Research Part B 43 (1), 119–126. Yang, H., Huang, H., 2005. Mathematical and Economic Theory of Road Pricing. Elsevier, Amsterdam. Yang, H., Meng, Q., Lee, D., 2004. Trial-and-error implementation of marginal-cost pricing on networks in the absence of demand functions. Transportation Research Part B 38 (6), 477–493. Yang, H., Xu, W., He, B., Meng, Q., 2010. Road pricing for congestion control with unknown demand and cost functions. Transportation Research Part C 18 (2), 157–175. Yang, H., Xu, W., Meng, Q., 2005. A sequential experimental approach for analyzing second-best road pricing with unknown demand functions. In: Mahmassani, H.S. (Ed.), Transportation and Traffic Theory 2005. Elsevier, pp. 23–42. Ye, H., Yang, H., 2013. Continuous price and flow dynamics of tradable mobility credits. Transportation Research Part B 57, 436–450. Zhang, D., Nagurney, A., 1996. On the local and global stability of a travel route choice adjustment process. Transportation Research Part B 30 (4), 245–262. Zhou, B., Bliemer, M., Yang, H., He, J., 2015. A trial-and-error congestion pricing scheme for networks with elastic demand and link capacity constraints. Transportation Research Part B 72, 77–92.

Please cite this article in press as: Ye, H., et al. Learning marginal-cost pricing via a trial-and-error procedure with day-to-day flow dynamics. Transportation Research Part B (2015), http://dx.doi.org/10.1016/j.trb.2015.08.001