- Email: [email protected]
Selecting the best choice in the weighted secretary problem
EUROPEAN JOURNAL OF OPERATIONAL RESEARCH
ELSEVIER
European Journal of Operational Research 92 (1996) 135-147
Theory and Methodology
Selecting the best choice in the weighted secretary problem Young Hak Chun
*
Department of Information Systems and Decision Sciences, College of Business Administration, Louisiana State Uniuersity, Baton Rouge, LA 70803-6316, USA
Received March 1994; revised January 1995
Abstract In the sequential evaluation and selection problem with n applicants, we assume that a decision maker has some prior information about each applicant so that unequal weights may be assigned to each applicant according to his or her likelihood of being the best among all applicants. Assuming that the pre-assigned weights are available in advance, we derive the optimal selection strategy that maximizes the probability of selecting the best among all applicants. For the case where the decision maker is permitted to rearrange the sequence in which applicants are evaluated, we further propose a simple heuristic procedure to the problem of optimally ordering the sequence of evaluations. Based on a pairwise comparison matrix and a goal programming procedure, we also propose a method that easily computes the weights in a practical situation. Keywords: Decision theory; Search theory; Dynamic programming; Stochastic processes
1. Introduction In many decisions of everyday life such as buying a car, hiring an employee, or accepting a marriage proposal, several choices are presented sequentially over a period of time. Following an evaluation of a choice, the decision maker may either select the choice under consideration or reject it and consider the next one. The model corresponding to such decision situations under certain assumptions is commonly referred to as the secretary problem, candidate problem, parking spot problem, beauty contest problem, streetwalker problem, marriage problem, bachelor’s dilemma, or dowry problem.
* Corresponding
author. Fax: (504) 388-2511
The secretary problem, which appeared first in print in Chow et al. (1964), seems to be the most common name for the sequential evaluation and selection problem, in which one must make an irrevocable choice from a number of applicants whose values are revealed only sequentially. It is also a classic example of a problem amenable to a dynamic programming treatment. The structural assumptions made in the classical secretary problems can be summarized as follows (Ferguson, 1989): 1. There is a known number of applicants for one secretarial position available. 2. The applicants are interviewed sequentially in random order, each order being equally likely. 3. You can rank all the applicants from best to worst without ties. The decision to accept or reject an applicant must be based only on the
0377-2217/96/$15.00 0 1996 Elsevier Science B.V. All rights reserved SSDZ 0377-2217(95)00045-3
136
YH. Chun /European
Journal of Operational Research 92 (1996) 135-147
relative ranks of those applicants interviewed so far. 4. An applicant once rejected cannot later be recalled. 5. You are very particular and will be satisfied with nothing but the very best. Under these assumptions, the question is when to stop the evaluation process and select a candidate. If the evaluation process is terminated too early, the applicants better than the selected candidate may not have been interviewed yet; if the decision is made too late, the better candidates may have been rejected earlier in the evaluation process. Thus, the objective in the secretary problem is to derive the optimal selection strategy which maximizes the probability of selecting the best among all choices. Since its appearance in literature in the late 1950’s and early 1960’s, the secretary problem has been extended and generalized in many different directions by releasing some of the basic assumptions. Now one can say that it constitutes a “field” of study within mathematics-probability-optimization (Ferguson, 1989). As further evidenced by Freeman (19831, this particular field of study has experienced rapid growth and extensive application to vastly different problems. Although the literature in this area is extensive, there is no known effort to date which relaxed the second assumption: Before the evaluation process begins, each applicant has the same probability of being the best among all the applicants. In more realistic cases, however, an interviewer may have some prior information about each applicant, which enables the interviewer to rank the applicants according to their credentials or to assign a weight on each applicant. Before starting job interviews, for example, an employer or a search committee may assign unequal weights on all applicants based on the initial evaluation of their credentials. In this paper, we consider the weighted secretary problem, in which each choice is assigned an unequal weight based on its likelihood of being the best, and derive the optimal selection strategy which maximizes the probability of selecting the best choice. In the classical secretary problem, we assume that each applicant has an equal probabil-
ity of being the best, not allowing any prior information about each of the applicants. In the weighted secretary problem, however, each choice is assigned a weight wj, j = 1,2,. . . ,n, based on its likelihood of being the best among all II applicants. If each applicant has the equal weights (i.e., wj = w for j = 1,2,. . .,n), then it can be shown that the weighted secretary problem will reduce to the classical secretary problem. Thus, the weighted secretary problem can be considered as a more generalized version of the classical secretary problem. In the next section, we present some of the terminology and notation used throughout this paper, and discuss the basic assumption on which the optimal selection strategies for the weighted secretary problem are based. A dynamic programming formulation of the weighted secretary problem is presented in Section 3. Due to the “all-or-nothing” property of the weighted secretary problem, we can derive a remarkably simple selection strategy in Section 4 based on the Markov decision process. In Section 5, we propose a simple ordering strategy for the case where a decision maker is permitted to determine the sequence in which choices are evaluated. Section 6 deals with how to determine the weights based on a decision maker’s pairwise comparison matrix. Concluding remarks are contained in Section 7.
2. Notation and assumption In the weighted secretary problem, the set S, = {1,2,. . .) n) represents the order in which IZ applicants or, in more neural terms, choices are interviewed or evaluated. The size of a weighted secretary problem is defined as the number of choices in the sequence. For a given sequence of choices in S,, we define the jth stage of the evaluation process as the jth choice in the sequence to be evaluated. The best choice in the set S,, is called the absolutely best choice, while the best choice in the subset Sj = {1,2,. . . ,j) c S, is called a relatively best choice at stage j. Since we are interested in nothing but the best choice as in the classical secretary problem, we
Y.H. Chun /European
Suppose, for instance, we have two choices with wi = 1 and w2 = 3, respectively. Then, the probability that Choice 1 is better than Choice 2 is assumed to be l/(1 + 3), while the probability that Choice 2 outranks Choice 1 is 3/(1 + 3). We now devote the rest of this section to explaining the validity of the assumption, and discuss in Section 7 how to obtain the weights in a practical situation. When the probability distribution of rank orders is available, the exact solution can be obtained via a decision tree or a dynamic programming table without the simplifying assumption. Consider, for example, the secretary problem of size 3 in which there are 3! possible rank orders. Let P[i,j,k] or, more concisely, pijk be the probability that the rank order is i, j, and k, where i,j,k E (1,2,3}. For example, pzjL represents the probability that the first choice will be of rank 2, the second choice will be of rank 3, and the last choice will be the best choice among the three. When all the probabilities pijk are available, the evaluation process can be represented by a decision tree (see Clemen, 1991 about the decision tree, a graphical representation of the decisionmaking process). Note that the tree shows the natural or logical progression that will occur over time. First of all, for example, the decision-maker must decide whether to stop and select the first candidate or to go to the next stage in the secretary problem of size 3. If the first choice is selected, the winning probability is pIz3 + P,~~,
can terminate the selection process only with a relatively best choice; if not a relatively best choice, the choice can never be the absolutely best choice. Thus, a relatively best choice is referred to as a candidate and, if a candidate in the set Sj appears at the current stage j, the candidate is said to be available for selection. The secretary problem can be considered as a game in which we must select the absolutely best choice to win. The probability of winning or, more concisely, the winning probability is the probability of selecting the absolutely best choice. As in the classical secretary problem, the objective in the weighted secretary problem is to find the optimal selection strategy which maximizes the winning probability, r. w,}, where wj 2 0, be the Let W={w,,w,,..., set of weights assigned before evaluations to each choice according to their likelihood of being the absolutely best choice. For notational convenience, the cumulative weight up to and including the jth stage is given by ‘/;.= wr + w2 + . . . + wj, for j = 1,2,. . . ,n. Thus, the sum of the total weights assigned to IZ choices is V,. We assume in the weighted secretary problem that the weights are assigned to the choices such that the following assumption is satisfied: Assumption.
The probability that the ith choice with weight wi is better than any other choices in the set S, is wi/(wl + w2 + . . . +wj) = wi/lf., where 1
stop
137
Journal of Operational Research 92 (1996) 135-147
&13+Psi2 stop p213+p312+p321
h3?3+Pl32
/
/ F321
PIZX+FB~+PZN
Yes 1
F213+p312+p321
\NO
-
No
0
Yes 1 Go Fl??+p132\ F123+p132
Fig. 1. Decision
tree for the weighted
secretary
problem
+&l
with three choices.
No0
Y.H. Chun/European
138
Journal of Operational Research 92 (1996) 135-147
which is given as a payoff (or winning probability) at the end of the act branch with a square in Fig. 1. If the decision-maker decides to reject the first and go for the second choice, then the second choice may be either a relatively best choice or not. The corresponding probabilities, p2i3 + P312 + P321 and
PI23
+ P132
+ P231,
are
given
on
the
state brunches with a circle in Fig. 1.
When the second choice is a relatively best choice, the decision-maker may stop the evaluation process, receiving the payoff, (p213 + +p321). If the decision-maker p312)/(p213 +p312 decides to continue for the third choice, and the third choice turns out to be a relatively best choice, then the winning probability with the last choice will be 1, which means that the last choice will be the absolutely best choice among the three. If the second choice is not a relatively best choice (i.e., worse than the first choice), the decision-maker has no choice but to continue to the third stage. In that case, the third choice may be a relatively best choice with a probability p231/(p123 +p132 fp,,,) or not with a probability (pi23 +pi32)/(&, +1)132 +P231) as shown in Fig. 1. The optimal selection strategy can be obtained by working backward in the decision tree in Fig. 1.
The decision tree may be represented by a table as in Table 1, based on which we can obtain the exact solution more conveniently without the simplifying assumption. In the stochastic dynamic programming formulation of the weighted secretary problem (see Ross, 1983 for more details about the stochastic dynamic programming), the decision set is {Stop, Go} and the set of possible states is {Yes,No}. The stage represents the sequence of applicants as explained before. Let rs(j> denote the winning probability obtained when we select the currently available candidate and stop the selection process at stage j. On the other hand, let r&j) be the winning probability obtained when we decide to continue the evaluation process at stage j. Then, the optimal selection strategy at stage j when a candidate is available is simply to stop the process if r&j> 2 rC(j) and to continue otherwise. Thus, let 7 *(j) = max(rr,( j>,n-Ccj)} represent the m&mum winning probability obtainable with a candidate at stage j. It can be shown from Table 1 that, when we have three choices in the secretary problem, the optimal decision at the first stage, for example, is to select the first choice and terminate the evaluation process if p123 + P132 2 max{p213 + dynamic programming
Table 1 Dynamic programming table for the weighted secretary problem with three choices Stage
State
Yes
No
3
stop Go i-r”(3)
1 0 1
0 0 0 p213 +p312
stop
p213 +p312
0
+p321 p231
p321
Go
p213+p312 maX{pz13
p123 +pl32
+p321 +p312rp321
P*(2) p223 +p312
p123 +p132
+p321
_
stop
pl23
Go
max(pz,,
+p312rp321)
P*(l)
maxIpI
+P132,
+p132
_
+p231
m&p213
+p231
p231
1
+P312~P32J
+p
231)
+p231
YH. Chun/European
+ P231. The maximum winning probability obtainable when we follows the optimal selection strategy is maxipi*, + pij2, maxlp,,, +
P312&21)
P312~P3211+
139
Journal of Operational Research 92 (1996) 135-147
P23J
Since the possible number of rank orders in the secretary problem of size n is n!, it is practically impossible for a decision-maker to specify the probability distribution of the all rank orders if n is relatively large. When a decision-maker has ten applicants, for example, a total of 3.6 million probabilities must be specified to obtain the exact solution via a decision tree or a dynamic programming approach. That is the reason that we propose a simple selection strategy based on the simplifying assumption. The similar weighting scheme has been used previously in Schwertman et al. (1991). For the NCAA regional basketball tournaments, the probability of winning in a particular game is expressed as the opponents’ seeding divided by the sum of its own seeding plus that of the opponents; i.e., P[Seed i defeats Seed jl = j/(i + j). Based on the assumption, we derive the optimal selection strategies for the weighted secretary problem in the following sections. To provide further insight into the selection strategies discussed, we illustrate the selection strategies using the weighted secretary problem W = I1,3,2,3,1}, in which the weights on five applicants are 1, 3, 2, 3, and 1, respectively. We first cast the optimal selection problem as a dynamic programming and derive a recursive equation in the next section.
and stop the evaluation process. From the assumption, the conditional probability that the relatively best choice at stage j will be the absolutely best choice can be shown to be Q/V,. Thus, the winning probability if we stop at stage j and select the currently available candidate is
which is nondecreasing in j. Since the relatively best choice at the last stage is certainly the absolutely best choice, we have am = 1. Now consider the winning probability obtainable when the decision maker decides to continue to the next stage, rejecting the current candidate at stage j. Since the no-choice option (Sakaguchi, 1984) is not considered in this paper, we are not permitted to terminate the evaluation process without selecting a choice. Thus -rr,(n>= 0 and r*(n) = max(T&n),rc(n>} = 1 at the lust stage. Suppose that we have decided to continue at an intermediate stage j + n. From the assumption, the probability that another candidate will be available at the next stage j + 1 is w,+,/r/;+,. If so, we will obtain 7 *(j + 1) by following the optimal selection strategy at that stage j + 1. On the other hand, the probability that the next choice will not be a candidate is r/;../y + , . In such a case, we must continue the evaluation process and our gain will be n,( j + 1). Thus, the expected winning probability if we continue at stage j is expressed as a convex combination of r *(j + 1) and rcTT,( j + 1) as follows: wj+l
d0
--r*(j+l) v,
=
+ &rJj
In the weighted secretary problem, we have only two alternative actions at each stage: (1) to select the currently available candidate and terminate the evaluation process, or (2) to continue to the next stage. In weighing these two options at stage j, j = 1,2,. . . ,n, we must take into account the winning probabilities, rr&j) and rc(j), obtainable by each option. Suppose that Choice j turns out to be the relatively best choice in the subset Sj = (1,2,. . . ,j) G S,, and the decision maker decides to select it
+ I),
I+1
If1
3. Dynamic programming formulation
(1)
for j = 1,2,. . . ,n,
rss( j) = T./V,,
for j= 1,2 ,..., n - 1.
(2)
By solving (2) recursively, we have TJj)
=
PjkT*(k),
k
forj=1,2
,..., n-l,
k=j+l (3)
where pjk is the transition probability from stage j to stage k (Dynkin, 1963): Pjk = -
r/;.wk
v k-l
-
Vk’
forl
YH. Chun /European Journal of Operational Research 92 (1996) 135-147
140
Thus, it follows from (1) and (3) that r*(j)
Since we must compute the winning probabilities r$j) and n-&j> for each stage j = 1,2,. . . ,n, the calculation for the recursive equation in (5) is rather tedious and time-consuming for a weighted secretary problem with more than 5 or 6 choices. In the following section, we propose another selection strategy by other means which is remarkably simple in computation.
=max{rAj),rAj)}
I
forj= 1,2 ,..., n - 1, 1, forj=n. (5)
This optimality equation can be solved recursively by working backward from the last stage n and the optimal selection strategy at stage j is simply stated as follows: Selection Strategy I. For a given weighted secretary problem, compute rsTTs( j>, rc( j> and r*( j> recursively for every stage j = 129 ,***, n. Then, when a candidate is available at stage j, select the candidate if r *(j> = TJ j). Continue the evaluation process otherwise. By the principle of optimality in dynamic programming (see, e.g., Bellman and Dreyfus, 19621, the winning probability GT under the Selection Strategy I is equal to r*(l), which is the maximum winning probability at the starting stage. Example 1. Based on Table 2 which shows the winning probabilities at each stage, the optimal selection strategy for the weighted secretary problem W= {1,3,2,3,1) is stated as follows: Do not select the first or second choice because r&j) < rcTT,( j) for j = 1 and 2. Then, select the first candidate available thereafter because rr&j> > r,(j) for j = 3, 4, and 5. Under this selection strategy, the winning probability rr is r * (1) = 0.4444.
Table 2 Winning probabilities
at each stages in the weighted
i
wj
Y
ii-&i)
5 4 3 2 1
1 3 2 3 1
10 9 6 4 1
1.0 0.9 0.6 0.4 0.1
secretary
4. Markov decision process Since the objective in the weighted secretary problem is to find only the absolutely best choice, the weighted secretary problem can be considered as a best choice problem in which a decision-maker’s utility is equal to one if the selected choice is the absolutely best choice, and zero otherwise. Due to its special property of the “all-or-nothing” utility function in the best choice problem, the weighted secretary problem can be described as a two-action Markov decision process (Ross, 1970, p. 119), because both the payoffs or winning probabilities r&j> and rr,(j) and the transition probabilities pij in (4) are functions only of the last state and the subsequent action, which is either to continue to the next stage or to terminate the evaluation process at the current stage. Let the subsets A and B be defined such that A = {k I rc(k)
> rs(k)},
B={kl~Jk)
a+)],
(6)
where AnB=O and AUB=S,. In other words, A is a subset of stages at which it is worth continuing the evaluation process, while B is a
problem
W= (1,3,2,3,1)
?r,(j)
77*(i)
Decision
0.000 0.100 0.366 0.444 0.444
1.000 0.900 0.600 0.444 0.444
stop stop stop Go Go
I’.H. Chun /European
Journal of Operational Research 92 (1996) 135-147
subset of stages at which it is worth selecting a candidate. Thus, in the two-action Markov decision process, the optimal strategy is simply to identify the subsets A and B. In the weighted secretary problem, however, the subsets A and B can be easily identified due to its special property. The following lemma proves that, if it is not worth selecting a candidate at a stage, then it is also not worth selecting a candidate at any earlier stages. Lemmal.
ZfjEA,
thenj-lEAforj=2,3
,...,
n.
Proof. See Appendix A. On the other hand, the following lemma states that, if it is worth selecting a candidate at stage j, then it is also worth selecting a candidate at a later stage. Lemma 2. Zf i E B, then i + 1 E B for i = 1,2,. . . , n - 1.
Fig. 2. Determination
141
Proof. It is obvious from Lemma 1 since the subset B is defined as the complement of the subset A. 0 Lemma 1 and Lemma 2 together imply that, in the weighted secretary problem, the set S, = , , . . . ,n} can be bisected into two disjoint sets 112 A={12 , >***, j* -11 and B=(j*,j* +l,...,n} where j* is defined as the starting stage in the second subset B. This also implies that the weighted secretary problem satisfies the monotone condition as in the classical secretary problem (Chow et al., 19711, and thus the one-stage look-ahead COLA) policy (Ross, 1970) is optimal. Let a( j * ) be the maximum winning probability obtainable when we optimally bisect the sequence S, into A = {1,2,. . . ,j * - 1) and B = {j*,j* + 1 , . . . ,n}. Since the subset B is defined as a set of stages k at which r&k) I r&k), we have 7 *(k) = rssr,(k) for k E B. Thus, it follows from (1) and (4) that the winning probability rc(i) obtainable by continuing the evaluation process at stage i = j * - 1, j *, . . . ,n becomes
of the optimal
value j *.
142
IX
Tc(i) = t
Chun /European
Journal of Operational Research 92 (1996) 135-147
selection strategy, the maximum winning probability r( j *) is given in (8).
PikrsCk)
k=i+l
(7) It follows from (7) that the maximum winning probability rr( j * 1 obtainable when we follow the “divide-and-select” strategy with j * from the beginning of the evaluation process is expressed as: r(j*)
=rr*(l)
=rC(l)
=rr,(2)
= .-*
=rc(j* - 1) =
Suppose that, in the weighted secretary problem, the weights are the same: wj = w for Vj E S,. Then, V, = kw and the optimality inequality condition in Theorem 1 reduces to
(10) and the winning probability in (8) also reduces to
(8)
Since the optimal selection strategy is to select the candidate available for the first time in the second subset B, it is clearly suboptimal to select a candidate in the first subset A or to reject a candidate in the second subset B. Thus, it is obvious that r( j * ) = max(r( j) for Vj E S,}. If the “divide-and-select” strategy is optimal, then the problem is how to identify the starting point j * of the second set B for a given sequence of weights. In other words, we must determine from which stage j * we are ready to make a decision when we encounter a candidate. The optimal value j * is obtained from following theorem and illustrated in Fig. 2: Theorem 1. In the optimally divided subsets A = (12, ,**.7 j*11 and B={j*j*+l,...,n), the value j * satisfies the following inequality:
Proof. See Appendix B.
J;*
Selection Strategy II. Divide the sequence of choices into two disjoint sets, A = (1 32 ,*.., j * - 11 and B = (j *,j * + 1, , . . ,n}, where j * is obtained from Theorem 1. Then select the candidate available for the first time in the subset B. Under the
1
n
1
=eks*k-l.
(11)
These results are consistent with those of the classical secretary problem of size 12(Gilbert and Mosteller, 1966). Example 2. In the weighted secretary problem W = {1,3,2,3,1}, the optimal value of j is j * = 3 because it satisfies the optimality inequality condition in Theorem 2 as follows: $+;+~>I>;+$ 2
4
3 =a
i
+
5 +
f
3 =
4
1.111> 12 0.611= 5 + $.
Thus, the sequence is divided into two disjoint sets: A = {1,2) and B = {3,4,5), and the optimal selection strategy is to choose the first available candidate at or after the third stage. This strategy is consistent with that in Example 1. It follows from (8) that the winning probability obtained under this selection strategy is T(3)
Based on j* obtained from Theorem 1, the optimal selection strategy which depends only on j * is stated as follows:
-
= A( f + $ + 4) = 0.4444,
which is also consistent with the result in Example 1.
5. Optimal ordering of choices
The weighted secretary problem gives rise to another interesting decision problem if we are
Y.H. Chun /European
Journal of Operational Research 92 (1996) 135-147
permitted to determine the sequence in which choices are evaluated. Here the decision is to select the order of choices which maximizes the likelihood of selecting the best choice. We call this problem the optimal ordering problem. A complete enumeration could be used to determine the optimal ordering strategy: For each sequence, compute the winning probability in (8) and then select the sequence with the highest one. Clearly, this method is unacceptable when the problem size is relatively large, because the total number of possible orderings for n choices of different weights is n! For instance, we have more than 3.6 million possible orderings for just 10 choices of different weights. We could formulate the optimal ordering problem as a nonlinear assignment problem. However, an increase in the number of choices in the optimal ordering problem significantly increases the number of variables and constraints in this assignment problem. Such a problem is known and referred to in dynamic programming as the curse of dimension&y (Bellman and Dreyfus, 1962). As an alternative, we develop a simple heuristic procedure based on the following theorem, which states that the choices in the second subset B should be arranged in non-increasing order of their weights: Theorem 2. The ordering in the subset B is optimal ifwizwi+, whereiEB. Proof. See Appendix C.
Note that the sequence of choices in the first subset A does not make any difference in the winning probability because any choices in A will not be selected anyway under the selection strategy II. Based on Theorem 2, we can develop a simple heuristic procedure to the ordering problem as follows: Strategy. According to Selection Strategy II, divide the original sequence of choices into two disjoint sets, A = (1 92,.*., j * - l} and B={j*,j* + 1,...,n) where j* is obtained from Ordering
143
Table 3 Winning probabilities for various orderings in the weighted secretary problem W = (1137273 >1) Ordering in B
Winning probability
(3,4,5) (3,5,4) (4,3,5) {4,5,3) I5,3,41 I5,4,3)
0.4444 0.4381 0.4587 0.4571 0.4314 0.4400
Theorem 1. Then, rearrange the choices in the second subset B in non-increasing order of their weights. Example 3. In Example 2, the choices have been divided into the two disjoint sets, A = {1,2} and B = {3,4,5) and the winning probability in this case was 0.444. If we are permitted to rearrange the order of choices, the winning probability can be increased according to Theorem 2. Table 3 shows the winning probabilities for various orderings of choices in the subset B. From Table 3, we can verify that the ordering in the set B is optimal if the choices in B are arranged in the decreasing order of their weights; i.e., B = {4,3,5) gives the highest winning probability, 0.4587.
6. Obtaining
weights for each choice
Based on the simplifying assumption about the relationship between the weights and the winning probability in Section 2, we have derived the optimal selection strategy in Section 3 and Section 4 and an ordering strategy in Section 5 for the weighted secretary problem. Now we are in a position to discuss about how to obtain the weights for each choice in a practical situation. In this section, we propose a simple weighting scheme that is based on a pairwise comparison matrix and a goal programming procedure. Suppose there are n choices. We begin by writing down an n x n matrix (known as the pairwise comparison matrix) Q. The element qi, in row i and column j of Q is the probability that
YH. Chun /European
144
Journal
of Operational Rest arch
the ith choice is better than the jth choice. For all i, it is necessary that qii = 1. If, for example, 4 i3 = 0.3, the probability that the first choice is better than the third choice in the sequence is 0.3. For consistency, it is necessary that qji = 1 -
To satisfy the assumption in Section 2, the weights must be determined such that wi
___ wi
Suppose that the pair-wise comparison matrix Q is available for the weighted secretary problem.
for i #j.
qij,
TO 21
3j 4)
;;
LP
= wj
S12 + 513 + S14 + S15 + S21 + S23 + S24 + S25 + S31 + S32 + 534 + 535 + 541 + S42 + S43 + S45 + S51 + S52 + S53 + S54
SUBJECT
END
+
9) 10) 11) 12) OPTIMUM
S12 - S21 + S13 - S31 + s14 - S41 + s15 - s51 + S23 - 532 + 524 - 542 + S25 - S52 + s34 - S43 + s35 - s53 + 545 - 554 + Wl + w2 + w3
0.75 Wl 0.67 Wl 0.75 wl 0.50 Wl 0.40 W2 0.50 W2 0.25 w2 0.60 w3 0.37 w3 0.25 W4 + w4 +
FOUND AT STEP
11
OBJECTIVE 1) VARIABLE s12 s13 s14 515 s21 S23 S24
0.25 0.33 0.25 0.50 0.60 0.50 0.75 0.40 0.66 0.25 =
0
W2 = W3 = W4 =
0 0
w5 = w3 = w4 = w5 = w4 = w5 = w5 = 10
.590000100
.oooooo .oooooo .oooooo .oooooo .oooooo
S25 531
.oooooo .010000
S32 s34 s35 s41 S42 s43 s45 551 S52
.oooooo .oooooo
s53 554 Wl
w5
FUNCTION VALUE
VALUE .oooooo .oooooo
.oooooo .oooooo .oooooo
.oooooo .oooooo .oooooo .oooooo
REDUCED COST 1.477333 2 .oooooo .oooooo
.562000 .522667 .968333 1.882000
.oooooo .oooooo 1.031667 1 .oooooo 2.000000 2.000000 .118000 1 .oooooo 2.000000 1.438000 2 .oooooo
.080000 .500000 1.000000
.oooooo .oooooo .oooooo
w2 w3
3.000000 2.000000
.oooooo .oooooo
w4 w5
3.000000 1.000000
.oooooo .oooooo
Fig. 3. LINDO
solution
(12)
If there is any inconsistency between the pairwise comparison matrix and the weights, it can be minimized via a goal programming approach
qij.
MIN
92 (1996) 135-147
to the goal programming
formulation.
Y.H. Chun /European
(Ignizio, 1976). With the deviational variables sij, the appropriate goal programming formulation would be min 2 = C sij
(13)
i#j
AHP, and Zahedi (1986) and Saaty (1990) for a discussion of applications of AHP. Example 4. Suppose a decision-maker’s
pairwise comparison matrix for five applicants is given as follows:
s.t. qjiwi - qijwj = sji - sij, w,+w,+
for all i fj,
**. +w,=10,
(14)
(15)
all variables nonnegative. The last constraint (15) is necessary to exclude the trivial solution, wi = 0 for all i. When the weighting scheme is not consistent with the pairwise comparison matrix Q, an element in Q can be expressed with an error term as follows: wi + Aij qii = ___ wi + w,
for i #j,
(16)
where -CC < A,, < + 03 and Aij + Aji = 0. Thus, in the goal programming formulation, it can be shown from (16) that the constraints in (14) is expressed as wj - Ajj wi + Aij
sji - sij = -w. wi+wj
- -w. I wi + wj
’
=~{(Wj-Aij)~,-(Wi+dij)Wj) I J
= Aji.
(17)
Thus, the objective function value in (13) is expressed as minz= =
Csjj=
C(sij+sji)
i#j
i
C i
(‘ji\T
145
Journal of Operational Research 92 (1996) 135-147
(18)
which implies that, if the weights obtained by the goal programming are perfectly consistent with the decision-maker’s pairwise comparison matrix, then the objective function value becomes zero. In that sense, the objective function value in the goal programming formulation can be interpreted as the consktency index as in Thomas Saaty’s analytic hierarchy process @HP). See the special issue of European Journal of Operational Research (Vol. 48, No. 1, 1990) for more details on
Q=
1 0.75
0.25 1
0.67 0.75 I 0.50
0.40 0.50 0.25
0.33 0.60 1
0.25 0.50
0.50 0.75
0.4
0.60 0.34
1
0.66 0.75
0.25
1
Based on the pair-wise comparison matrix, we formulated the problem as a goal programming and solved it using LINDO in Fig. 3. According to Fig. 3, the optimal solution is W= (1,3,2,3,1}, which has been used in the previous examples. In fact, the pairwise comparison matrix was derived arbitrarily from the weight W = (1,3,2,3,1}. However, due to the rounding-off error in the pair-wise comparison matrix, the objective function value or the consistency index is not zero in Fig. 3. 7. Concluding remarks For the weighted secretary problem, we have derived the optimal selection strategies under which the probability of selecting the best choice is maximized. We have also considered an ordering strategy for the case where the decision maker can rearrange the sequence of choices. Our results suggest several possible directions for further investigation. Some of these efforts, regarding the weighted secretary problem, are currently being developed and include consideration of (1) selecting more than one candidate, (2) using the maximum entropy criterion, (3) evaluating choices using multiple criteria, and (4) assuming that the number of applicants is not known in advance. The classical secretary problem and its extensions have been successfully applied to a wide variety of different situations arising in operational research applications. Since the weighted secretary problem is a more generalized version of the classical secretary problem, the weighted secretary problem may be applied to most areas where the classical secretary problem has been
YH. Chun /European
146
Journal of Operational Research 92 (1996) 135-147
successfully applied. Some of the possible applications are in consumer choice behavior (Nelson, 1970 and Rapoport and Tversky, 19701, job search (Lippman and McCall, 1976 and McCall, 19651, stock price prediction (Hlynka and Sheahan, 1988), sequential assignment of batched jobs (Albright, 1974; Albright and Derman, 1972; Derman et al., 1972), the house selling problem (Albright, 1977), and in searching for the lowest price product or service (Telser, 1973 and Rothschild, 1974).
which implies that 12
i
wk
k=j*+l
*
‘k-1
It also follows from (8) that q*-2
-l)=v
’
n =-
wk
c k=;*-1 vk-l
y*_,
K i Wj*-1
Appendix A. Proof of Lemma 1
--
K
By (61, this lemma states that, if rc(j) > r&j>, then r,( j - 1) > r&j - 1) for j = 2,3,. . . ,n. The hypothesis rrcTT,( j) > rsTTs( j> implies that r *(j) = r&j). Meanwhile, it follows from (2) with the value j reduced by one that v, r,(j),
which implies that
k
I
I
n
,$*+=
Y-1
rc(j-l)=$rr*(j)+-
wj*-l
=7r(ji)-7
which is rcTT,( j - 1) = r&j> because r *(j) = TG(j) from the hypothesis. Thus, the hypothesis r&j) > rs(j> implies that rc(j - 1) > r&j). Since rs(j> is increasing in j as shown in (11, we see that rs,( j) > rsTT,( j - 1). Thus, we conclude rcTT,( j - 1) > rJj - 1). ? ?
??
1
Appendix C. Proof of Theorem 2 Suppose that wi < wi+ 1 where i E B. Then, it follows from (8) that the winning probability in this sequence is
Appendix B. Proof of Theorem 1 Note that rr( j * ) = max(rr( j) for Vj E S,). It follows from (8) that
+
h wk
p(j*+l)=S+
n
k=j'+l
‘k-1 wi* \
If we interchange the two choices at stages i and i + 1, then the winning probability in this rearranged sequence becomes
Y.H. Chun /European
The difference
Journal of Operational Research 92 (1996) 135-147
in the winning probabilities wi
wi+1
K-1 +
--
K-1 + wi+1
Bellman, R., and Dreyfus, S. (19621, Applied Dynamic Programming, Princeton University Press, Princeton, N.J. Chow, Y.S., Moriguti, S., Robbins, H., and Samuels, SM. (19641, “Optimal selection based on relative rank (the ‘secretary problem’)“, Israel Journal of Mathematics 2,
is
wi
K-1
81-90.
wi+1
Chow, Y.S., Robbins, H., and Siegmund, D. (19711, Great Expectations: The Theory of Optimal Stopping, Houghton Mifflin, Boston. Clemen, R.T. (1991), Making Hard Decisions: An Introduction to Decision Analysis, PWS-Kent, Boston. Derman, C., Lieberman, G.J., and Ross, S.M. (19721, “A sequential stochastic assignment problem”, Management Science 18, 349-355. Dynkin, E.B. (1963), “The optimum choice of the instant for stopping a Markov process”, Soviet Mathematics Doklady
r/;-, + Wi =-4-l K
wi+l
wi
wi
K-1
+
W z+l
-
K-1
+wi+1 wi
w,
l/_,+wi
-
I/;._l+wi
147
1
4, 627-629.
Ferguson, T.S. (1989), “Who solved the secretary problem?“,
1
1
[
-Wi
y-1wi( r ~_1+Wi
wi+l
zz-
K
y-1(
-
If-1
-
5-1
=-
K
(K-1
+
wi(wi+l
+
-
Ii
wi)
Journal 61, 35-73.
Wj)
wi(wi+l -
I/;:-l+wi+l
wi)(V,-l
-
Hlynka, M., and Sheahan, J.N. (19881, “The secretary problem for a random walk”, Stochastic Processes and Their
wi) +
Applications 28, 317-325. wi+l)
wi)
(K-l+wi)
1 -x i K-1
Statistical Science 4, 282-296.
Freeman, P.R. (1983), “The secretary problem and its extensions: A review”, International Statistical Review 51, 189206. Gilbert, J.P., and Mosteller, F. (19661, “Recognizing the maximum of a sequence”, American Statistical Association
1 (K-1 +wwi+1) 1
Ignizio, J.P. (19761, Goal Programming and Extensions, Lexington Books, Lexington, MA. Lippman, S.A., and McCall, J.J. (19761, “The economics of job search: A survey”, Economic Inquiry 14, 155-189. McCall, J.J. (19651, “The economics of information and optimal stopping rules”, Journal of Business 38, 300-317. Nelson, P. (1970), “Information and consumer behavior”, Journal of Political Economy 78, 311-329.
-
Rapoport, A., and Tversky, A. (19701, “Choice behavior in an optimal stopping task”, Organizational Behavior and Hu-
which implies that we can increase the winning probability by rearranging the choices in B in non-increasing order of their weights. ? ?
Ross, S.M. (19701, Applied Probability Models with Optimization Applications, Holden-Day, San Francisco. Ross, SM. (19831, Introduction to Stochastic Dynamic Programming, Academic Press, San Diego. Rothschild, M. (1974), “Searching for the lowest price when the distribution of prices is unknown”, Journal of Political
=-“;.-I
v,
wi) Wifl (K-1 +wi) K-dvi-1 +?+I)
wi(wi+l
> 0,
man Performance
5, 105-120.
Economy 82, 689-711.
References Albright, SC. (19771, “A Bayesian approach to a generalized housing selling problem”, Management Science 24, 432440.
Albright, S.C. (1974), “Optimal sequential assignments with random arrival times”, Management Science 21, 60-67. Albright, S.C., and Derman, C. (19721, “Asymptotic optimal policies for the stochastic sequential assignment problem”, Management Science 18, 46-51.
Saaty, T. (1990), Multicriteria decision making: The analytic hierarchy process, RWS Publications, Pittsburgh, PA. Sakaguchi, M. (19841, “Bilateral sequential games related to the no-information secretary problem”, Mathematics Japonica 29, 961-973.
Schwertman, N.C., McCready, T.A., and Howard, L. (19911, “Probability models for the NCAA regional basketball tournaments”, American Statistician 45, 35-38. Telser, L.G. (19731, “Searching for the lowest price”, American Economic Review LXIII, 40-49. Zahedi, F. (19861, “The analytic hierarchy process - A survey of the method and its applications”, Interfaces 16, 96-108.
ELSEVIER
European Journal of Operational Research 92 (1996) 135-147
Theory and Methodology
Selecting the best choice in the weighted secretary problem Young Hak Chun
*
Department of Information Systems and Decision Sciences, College of Business Administration, Louisiana State Uniuersity, Baton Rouge, LA 70803-6316, USA
Received March 1994; revised January 1995
Abstract In the sequential evaluation and selection problem with n applicants, we assume that a decision maker has some prior information about each applicant so that unequal weights may be assigned to each applicant according to his or her likelihood of being the best among all applicants. Assuming that the pre-assigned weights are available in advance, we derive the optimal selection strategy that maximizes the probability of selecting the best among all applicants. For the case where the decision maker is permitted to rearrange the sequence in which applicants are evaluated, we further propose a simple heuristic procedure to the problem of optimally ordering the sequence of evaluations. Based on a pairwise comparison matrix and a goal programming procedure, we also propose a method that easily computes the weights in a practical situation. Keywords: Decision theory; Search theory; Dynamic programming; Stochastic processes
1. Introduction In many decisions of everyday life such as buying a car, hiring an employee, or accepting a marriage proposal, several choices are presented sequentially over a period of time. Following an evaluation of a choice, the decision maker may either select the choice under consideration or reject it and consider the next one. The model corresponding to such decision situations under certain assumptions is commonly referred to as the secretary problem, candidate problem, parking spot problem, beauty contest problem, streetwalker problem, marriage problem, bachelor’s dilemma, or dowry problem.
* Corresponding
author. Fax: (504) 388-2511
The secretary problem, which appeared first in print in Chow et al. (1964), seems to be the most common name for the sequential evaluation and selection problem, in which one must make an irrevocable choice from a number of applicants whose values are revealed only sequentially. It is also a classic example of a problem amenable to a dynamic programming treatment. The structural assumptions made in the classical secretary problems can be summarized as follows (Ferguson, 1989): 1. There is a known number of applicants for one secretarial position available. 2. The applicants are interviewed sequentially in random order, each order being equally likely. 3. You can rank all the applicants from best to worst without ties. The decision to accept or reject an applicant must be based only on the
0377-2217/96/$15.00 0 1996 Elsevier Science B.V. All rights reserved SSDZ 0377-2217(95)00045-3
136
YH. Chun /European
Journal of Operational Research 92 (1996) 135-147
relative ranks of those applicants interviewed so far. 4. An applicant once rejected cannot later be recalled. 5. You are very particular and will be satisfied with nothing but the very best. Under these assumptions, the question is when to stop the evaluation process and select a candidate. If the evaluation process is terminated too early, the applicants better than the selected candidate may not have been interviewed yet; if the decision is made too late, the better candidates may have been rejected earlier in the evaluation process. Thus, the objective in the secretary problem is to derive the optimal selection strategy which maximizes the probability of selecting the best among all choices. Since its appearance in literature in the late 1950’s and early 1960’s, the secretary problem has been extended and generalized in many different directions by releasing some of the basic assumptions. Now one can say that it constitutes a “field” of study within mathematics-probability-optimization (Ferguson, 1989). As further evidenced by Freeman (19831, this particular field of study has experienced rapid growth and extensive application to vastly different problems. Although the literature in this area is extensive, there is no known effort to date which relaxed the second assumption: Before the evaluation process begins, each applicant has the same probability of being the best among all the applicants. In more realistic cases, however, an interviewer may have some prior information about each applicant, which enables the interviewer to rank the applicants according to their credentials or to assign a weight on each applicant. Before starting job interviews, for example, an employer or a search committee may assign unequal weights on all applicants based on the initial evaluation of their credentials. In this paper, we consider the weighted secretary problem, in which each choice is assigned an unequal weight based on its likelihood of being the best, and derive the optimal selection strategy which maximizes the probability of selecting the best choice. In the classical secretary problem, we assume that each applicant has an equal probabil-
ity of being the best, not allowing any prior information about each of the applicants. In the weighted secretary problem, however, each choice is assigned a weight wj, j = 1,2,. . . ,n, based on its likelihood of being the best among all II applicants. If each applicant has the equal weights (i.e., wj = w for j = 1,2,. . .,n), then it can be shown that the weighted secretary problem will reduce to the classical secretary problem. Thus, the weighted secretary problem can be considered as a more generalized version of the classical secretary problem. In the next section, we present some of the terminology and notation used throughout this paper, and discuss the basic assumption on which the optimal selection strategies for the weighted secretary problem are based. A dynamic programming formulation of the weighted secretary problem is presented in Section 3. Due to the “all-or-nothing” property of the weighted secretary problem, we can derive a remarkably simple selection strategy in Section 4 based on the Markov decision process. In Section 5, we propose a simple ordering strategy for the case where a decision maker is permitted to determine the sequence in which choices are evaluated. Section 6 deals with how to determine the weights based on a decision maker’s pairwise comparison matrix. Concluding remarks are contained in Section 7.
2. Notation and assumption In the weighted secretary problem, the set S, = {1,2,. . .) n) represents the order in which IZ applicants or, in more neural terms, choices are interviewed or evaluated. The size of a weighted secretary problem is defined as the number of choices in the sequence. For a given sequence of choices in S,, we define the jth stage of the evaluation process as the jth choice in the sequence to be evaluated. The best choice in the set S,, is called the absolutely best choice, while the best choice in the subset Sj = {1,2,. . . ,j) c S, is called a relatively best choice at stage j. Since we are interested in nothing but the best choice as in the classical secretary problem, we
Y.H. Chun /European
Suppose, for instance, we have two choices with wi = 1 and w2 = 3, respectively. Then, the probability that Choice 1 is better than Choice 2 is assumed to be l/(1 + 3), while the probability that Choice 2 outranks Choice 1 is 3/(1 + 3). We now devote the rest of this section to explaining the validity of the assumption, and discuss in Section 7 how to obtain the weights in a practical situation. When the probability distribution of rank orders is available, the exact solution can be obtained via a decision tree or a dynamic programming table without the simplifying assumption. Consider, for example, the secretary problem of size 3 in which there are 3! possible rank orders. Let P[i,j,k] or, more concisely, pijk be the probability that the rank order is i, j, and k, where i,j,k E (1,2,3}. For example, pzjL represents the probability that the first choice will be of rank 2, the second choice will be of rank 3, and the last choice will be the best choice among the three. When all the probabilities pijk are available, the evaluation process can be represented by a decision tree (see Clemen, 1991 about the decision tree, a graphical representation of the decisionmaking process). Note that the tree shows the natural or logical progression that will occur over time. First of all, for example, the decision-maker must decide whether to stop and select the first candidate or to go to the next stage in the secretary problem of size 3. If the first choice is selected, the winning probability is pIz3 + P,~~,
can terminate the selection process only with a relatively best choice; if not a relatively best choice, the choice can never be the absolutely best choice. Thus, a relatively best choice is referred to as a candidate and, if a candidate in the set Sj appears at the current stage j, the candidate is said to be available for selection. The secretary problem can be considered as a game in which we must select the absolutely best choice to win. The probability of winning or, more concisely, the winning probability is the probability of selecting the absolutely best choice. As in the classical secretary problem, the objective in the weighted secretary problem is to find the optimal selection strategy which maximizes the winning probability, r. w,}, where wj 2 0, be the Let W={w,,w,,..., set of weights assigned before evaluations to each choice according to their likelihood of being the absolutely best choice. For notational convenience, the cumulative weight up to and including the jth stage is given by ‘/;.= wr + w2 + . . . + wj, for j = 1,2,. . . ,n. Thus, the sum of the total weights assigned to IZ choices is V,. We assume in the weighted secretary problem that the weights are assigned to the choices such that the following assumption is satisfied: Assumption.
The probability that the ith choice with weight wi is better than any other choices in the set S, is wi/(wl + w2 + . . . +wj) = wi/lf., where 1
stop
137
Journal of Operational Research 92 (1996) 135-147
&13+Psi2 stop p213+p312+p321
h3?3+Pl32
/
/ F321
PIZX+FB~+PZN
Yes 1
F213+p312+p321
\NO
-
No
0
Yes 1 Go Fl??+p132\ F123+p132
Fig. 1. Decision
tree for the weighted
secretary
problem
+&l
with three choices.
No0
Y.H. Chun/European
138
Journal of Operational Research 92 (1996) 135-147
which is given as a payoff (or winning probability) at the end of the act branch with a square in Fig. 1. If the decision-maker decides to reject the first and go for the second choice, then the second choice may be either a relatively best choice or not. The corresponding probabilities, p2i3 + P312 + P321 and
PI23
+ P132
+ P231,
are
given
on
the
state brunches with a circle in Fig. 1.
When the second choice is a relatively best choice, the decision-maker may stop the evaluation process, receiving the payoff, (p213 + +p321). If the decision-maker p312)/(p213 +p312 decides to continue for the third choice, and the third choice turns out to be a relatively best choice, then the winning probability with the last choice will be 1, which means that the last choice will be the absolutely best choice among the three. If the second choice is not a relatively best choice (i.e., worse than the first choice), the decision-maker has no choice but to continue to the third stage. In that case, the third choice may be a relatively best choice with a probability p231/(p123 +p132 fp,,,) or not with a probability (pi23 +pi32)/(&, +1)132 +P231) as shown in Fig. 1. The optimal selection strategy can be obtained by working backward in the decision tree in Fig. 1.
The decision tree may be represented by a table as in Table 1, based on which we can obtain the exact solution more conveniently without the simplifying assumption. In the stochastic dynamic programming formulation of the weighted secretary problem (see Ross, 1983 for more details about the stochastic dynamic programming), the decision set is {Stop, Go} and the set of possible states is {Yes,No}. The stage represents the sequence of applicants as explained before. Let rs(j> denote the winning probability obtained when we select the currently available candidate and stop the selection process at stage j. On the other hand, let r&j) be the winning probability obtained when we decide to continue the evaluation process at stage j. Then, the optimal selection strategy at stage j when a candidate is available is simply to stop the process if r&j> 2 rC(j) and to continue otherwise. Thus, let 7 *(j) = max(rr,( j>,n-Ccj)} represent the m&mum winning probability obtainable with a candidate at stage j. It can be shown from Table 1 that, when we have three choices in the secretary problem, the optimal decision at the first stage, for example, is to select the first choice and terminate the evaluation process if p123 + P132 2 max{p213 + dynamic programming
Table 1 Dynamic programming table for the weighted secretary problem with three choices Stage
State
Yes
No
3
stop Go i-r”(3)
1 0 1
0 0 0 p213 +p312
stop
p213 +p312
0
+p321 p231
p321
Go
p213+p312 maX{pz13
p123 +pl32
+p321 +p312rp321
P*(2) p223 +p312
p123 +p132
+p321
_
stop
pl23
Go
max(pz,,
+p312rp321)
P*(l)
maxIpI
+P132,
+p132
_
+p231
m&p213
+p231
p231
1
+P312~P32J
+p
231)
+p231
YH. Chun/European
+ P231. The maximum winning probability obtainable when we follows the optimal selection strategy is maxipi*, + pij2, maxlp,,, +
P312&21)
P312~P3211+
139
Journal of Operational Research 92 (1996) 135-147
P23J
Since the possible number of rank orders in the secretary problem of size n is n!, it is practically impossible for a decision-maker to specify the probability distribution of the all rank orders if n is relatively large. When a decision-maker has ten applicants, for example, a total of 3.6 million probabilities must be specified to obtain the exact solution via a decision tree or a dynamic programming approach. That is the reason that we propose a simple selection strategy based on the simplifying assumption. The similar weighting scheme has been used previously in Schwertman et al. (1991). For the NCAA regional basketball tournaments, the probability of winning in a particular game is expressed as the opponents’ seeding divided by the sum of its own seeding plus that of the opponents; i.e., P[Seed i defeats Seed jl = j/(i + j). Based on the assumption, we derive the optimal selection strategies for the weighted secretary problem in the following sections. To provide further insight into the selection strategies discussed, we illustrate the selection strategies using the weighted secretary problem W = I1,3,2,3,1}, in which the weights on five applicants are 1, 3, 2, 3, and 1, respectively. We first cast the optimal selection problem as a dynamic programming and derive a recursive equation in the next section.
and stop the evaluation process. From the assumption, the conditional probability that the relatively best choice at stage j will be the absolutely best choice can be shown to be Q/V,. Thus, the winning probability if we stop at stage j and select the currently available candidate is
which is nondecreasing in j. Since the relatively best choice at the last stage is certainly the absolutely best choice, we have am = 1. Now consider the winning probability obtainable when the decision maker decides to continue to the next stage, rejecting the current candidate at stage j. Since the no-choice option (Sakaguchi, 1984) is not considered in this paper, we are not permitted to terminate the evaluation process without selecting a choice. Thus -rr,(n>= 0 and r*(n) = max(T&n),rc(n>} = 1 at the lust stage. Suppose that we have decided to continue at an intermediate stage j + n. From the assumption, the probability that another candidate will be available at the next stage j + 1 is w,+,/r/;+,. If so, we will obtain 7 *(j + 1) by following the optimal selection strategy at that stage j + 1. On the other hand, the probability that the next choice will not be a candidate is r/;../y + , . In such a case, we must continue the evaluation process and our gain will be n,( j + 1). Thus, the expected winning probability if we continue at stage j is expressed as a convex combination of r *(j + 1) and rcTT,( j + 1) as follows: wj+l
d0
--r*(j+l) v,
=
+ &rJj
In the weighted secretary problem, we have only two alternative actions at each stage: (1) to select the currently available candidate and terminate the evaluation process, or (2) to continue to the next stage. In weighing these two options at stage j, j = 1,2,. . . ,n, we must take into account the winning probabilities, rr&j) and rc(j), obtainable by each option. Suppose that Choice j turns out to be the relatively best choice in the subset Sj = (1,2,. . . ,j) G S,, and the decision maker decides to select it
+ I),
I+1
If1
3. Dynamic programming formulation
(1)
for j = 1,2,. . . ,n,
rss( j) = T./V,,
for j= 1,2 ,..., n - 1.
(2)
By solving (2) recursively, we have TJj)
=
PjkT*(k),
k
forj=1,2
,..., n-l,
k=j+l (3)
where pjk is the transition probability from stage j to stage k (Dynkin, 1963): Pjk = -
r/;.wk
v k-l
-
Vk’
forl
YH. Chun /European Journal of Operational Research 92 (1996) 135-147
140
Thus, it follows from (1) and (3) that r*(j)
Since we must compute the winning probabilities r$j) and n-&j> for each stage j = 1,2,. . . ,n, the calculation for the recursive equation in (5) is rather tedious and time-consuming for a weighted secretary problem with more than 5 or 6 choices. In the following section, we propose another selection strategy by other means which is remarkably simple in computation.
=max{rAj),rAj)}
I
forj= 1,2 ,..., n - 1, 1, forj=n. (5)
This optimality equation can be solved recursively by working backward from the last stage n and the optimal selection strategy at stage j is simply stated as follows: Selection Strategy I. For a given weighted secretary problem, compute rsTTs( j>, rc( j> and r*( j> recursively for every stage j = 129 ,***, n. Then, when a candidate is available at stage j, select the candidate if r *(j> = TJ j). Continue the evaluation process otherwise. By the principle of optimality in dynamic programming (see, e.g., Bellman and Dreyfus, 19621, the winning probability GT under the Selection Strategy I is equal to r*(l), which is the maximum winning probability at the starting stage. Example 1. Based on Table 2 which shows the winning probabilities at each stage, the optimal selection strategy for the weighted secretary problem W= {1,3,2,3,1) is stated as follows: Do not select the first or second choice because r&j) < rcTT,( j) for j = 1 and 2. Then, select the first candidate available thereafter because rr&j> > r,(j) for j = 3, 4, and 5. Under this selection strategy, the winning probability rr is r * (1) = 0.4444.
Table 2 Winning probabilities
at each stages in the weighted
i
wj
Y
ii-&i)
5 4 3 2 1
1 3 2 3 1
10 9 6 4 1
1.0 0.9 0.6 0.4 0.1
secretary
4. Markov decision process Since the objective in the weighted secretary problem is to find only the absolutely best choice, the weighted secretary problem can be considered as a best choice problem in which a decision-maker’s utility is equal to one if the selected choice is the absolutely best choice, and zero otherwise. Due to its special property of the “all-or-nothing” utility function in the best choice problem, the weighted secretary problem can be described as a two-action Markov decision process (Ross, 1970, p. 119), because both the payoffs or winning probabilities r&j> and rr,(j) and the transition probabilities pij in (4) are functions only of the last state and the subsequent action, which is either to continue to the next stage or to terminate the evaluation process at the current stage. Let the subsets A and B be defined such that A = {k I rc(k)
> rs(k)},
B={kl~Jk)
a+)],
(6)
where AnB=O and AUB=S,. In other words, A is a subset of stages at which it is worth continuing the evaluation process, while B is a
problem
W= (1,3,2,3,1)
?r,(j)
77*(i)
Decision
0.000 0.100 0.366 0.444 0.444
1.000 0.900 0.600 0.444 0.444
stop stop stop Go Go
I’.H. Chun /European
Journal of Operational Research 92 (1996) 135-147
subset of stages at which it is worth selecting a candidate. Thus, in the two-action Markov decision process, the optimal strategy is simply to identify the subsets A and B. In the weighted secretary problem, however, the subsets A and B can be easily identified due to its special property. The following lemma proves that, if it is not worth selecting a candidate at a stage, then it is also not worth selecting a candidate at any earlier stages. Lemmal.
ZfjEA,
thenj-lEAforj=2,3
,...,
n.
Proof. See Appendix A. On the other hand, the following lemma states that, if it is worth selecting a candidate at stage j, then it is also worth selecting a candidate at a later stage. Lemma 2. Zf i E B, then i + 1 E B for i = 1,2,. . . , n - 1.
Fig. 2. Determination
141
Proof. It is obvious from Lemma 1 since the subset B is defined as the complement of the subset A. 0 Lemma 1 and Lemma 2 together imply that, in the weighted secretary problem, the set S, = , , . . . ,n} can be bisected into two disjoint sets 112 A={12 , >***, j* -11 and B=(j*,j* +l,...,n} where j* is defined as the starting stage in the second subset B. This also implies that the weighted secretary problem satisfies the monotone condition as in the classical secretary problem (Chow et al., 19711, and thus the one-stage look-ahead COLA) policy (Ross, 1970) is optimal. Let a( j * ) be the maximum winning probability obtainable when we optimally bisect the sequence S, into A = {1,2,. . . ,j * - 1) and B = {j*,j* + 1 , . . . ,n}. Since the subset B is defined as a set of stages k at which r&k) I r&k), we have 7 *(k) = rssr,(k) for k E B. Thus, it follows from (1) and (4) that the winning probability rc(i) obtainable by continuing the evaluation process at stage i = j * - 1, j *, . . . ,n becomes
of the optimal
value j *.
142
IX
Tc(i) = t
Chun /European
Journal of Operational Research 92 (1996) 135-147
selection strategy, the maximum winning probability r( j *) is given in (8).
PikrsCk)
k=i+l
(7) It follows from (7) that the maximum winning probability rr( j * 1 obtainable when we follow the “divide-and-select” strategy with j * from the beginning of the evaluation process is expressed as: r(j*)
=rr*(l)
=rC(l)
=rr,(2)
= .-*
=rc(j* - 1) =
Suppose that, in the weighted secretary problem, the weights are the same: wj = w for Vj E S,. Then, V, = kw and the optimality inequality condition in Theorem 1 reduces to
(10) and the winning probability in (8) also reduces to
(8)
Since the optimal selection strategy is to select the candidate available for the first time in the second subset B, it is clearly suboptimal to select a candidate in the first subset A or to reject a candidate in the second subset B. Thus, it is obvious that r( j * ) = max(r( j) for Vj E S,}. If the “divide-and-select” strategy is optimal, then the problem is how to identify the starting point j * of the second set B for a given sequence of weights. In other words, we must determine from which stage j * we are ready to make a decision when we encounter a candidate. The optimal value j * is obtained from following theorem and illustrated in Fig. 2: Theorem 1. In the optimally divided subsets A = (12, ,**.7 j*11 and B={j*j*+l,...,n), the value j * satisfies the following inequality:
Proof. See Appendix B.
J;*
Selection Strategy II. Divide the sequence of choices into two disjoint sets, A = (1 32 ,*.., j * - 11 and B = (j *,j * + 1, , . . ,n}, where j * is obtained from Theorem 1. Then select the candidate available for the first time in the subset B. Under the
1
n
1
=eks*k-l.
(11)
These results are consistent with those of the classical secretary problem of size 12(Gilbert and Mosteller, 1966). Example 2. In the weighted secretary problem W = {1,3,2,3,1}, the optimal value of j is j * = 3 because it satisfies the optimality inequality condition in Theorem 2 as follows: $+;+~>I>;+$ 2
4
3 =a
i
+
5 +
f
3 =
4
1.111> 12 0.611= 5 + $.
Thus, the sequence is divided into two disjoint sets: A = {1,2) and B = {3,4,5), and the optimal selection strategy is to choose the first available candidate at or after the third stage. This strategy is consistent with that in Example 1. It follows from (8) that the winning probability obtained under this selection strategy is T(3)
Based on j* obtained from Theorem 1, the optimal selection strategy which depends only on j * is stated as follows:
-
= A( f + $ + 4) = 0.4444,
which is also consistent with the result in Example 1.
5. Optimal ordering of choices
The weighted secretary problem gives rise to another interesting decision problem if we are
Y.H. Chun /European
Journal of Operational Research 92 (1996) 135-147
permitted to determine the sequence in which choices are evaluated. Here the decision is to select the order of choices which maximizes the likelihood of selecting the best choice. We call this problem the optimal ordering problem. A complete enumeration could be used to determine the optimal ordering strategy: For each sequence, compute the winning probability in (8) and then select the sequence with the highest one. Clearly, this method is unacceptable when the problem size is relatively large, because the total number of possible orderings for n choices of different weights is n! For instance, we have more than 3.6 million possible orderings for just 10 choices of different weights. We could formulate the optimal ordering problem as a nonlinear assignment problem. However, an increase in the number of choices in the optimal ordering problem significantly increases the number of variables and constraints in this assignment problem. Such a problem is known and referred to in dynamic programming as the curse of dimension&y (Bellman and Dreyfus, 1962). As an alternative, we develop a simple heuristic procedure based on the following theorem, which states that the choices in the second subset B should be arranged in non-increasing order of their weights: Theorem 2. The ordering in the subset B is optimal ifwizwi+, whereiEB. Proof. See Appendix C.
Note that the sequence of choices in the first subset A does not make any difference in the winning probability because any choices in A will not be selected anyway under the selection strategy II. Based on Theorem 2, we can develop a simple heuristic procedure to the ordering problem as follows: Strategy. According to Selection Strategy II, divide the original sequence of choices into two disjoint sets, A = (1 92,.*., j * - l} and B={j*,j* + 1,...,n) where j* is obtained from Ordering
143
Table 3 Winning probabilities for various orderings in the weighted secretary problem W = (1137273 >1) Ordering in B
Winning probability
(3,4,5) (3,5,4) (4,3,5) {4,5,3) I5,3,41 I5,4,3)
0.4444 0.4381 0.4587 0.4571 0.4314 0.4400
Theorem 1. Then, rearrange the choices in the second subset B in non-increasing order of their weights. Example 3. In Example 2, the choices have been divided into the two disjoint sets, A = {1,2} and B = {3,4,5) and the winning probability in this case was 0.444. If we are permitted to rearrange the order of choices, the winning probability can be increased according to Theorem 2. Table 3 shows the winning probabilities for various orderings of choices in the subset B. From Table 3, we can verify that the ordering in the set B is optimal if the choices in B are arranged in the decreasing order of their weights; i.e., B = {4,3,5) gives the highest winning probability, 0.4587.
6. Obtaining
weights for each choice
Based on the simplifying assumption about the relationship between the weights and the winning probability in Section 2, we have derived the optimal selection strategy in Section 3 and Section 4 and an ordering strategy in Section 5 for the weighted secretary problem. Now we are in a position to discuss about how to obtain the weights for each choice in a practical situation. In this section, we propose a simple weighting scheme that is based on a pairwise comparison matrix and a goal programming procedure. Suppose there are n choices. We begin by writing down an n x n matrix (known as the pairwise comparison matrix) Q. The element qi, in row i and column j of Q is the probability that
YH. Chun /European
144
Journal
of Operational Rest arch
the ith choice is better than the jth choice. For all i, it is necessary that qii = 1. If, for example, 4 i3 = 0.3, the probability that the first choice is better than the third choice in the sequence is 0.3. For consistency, it is necessary that qji = 1 -
To satisfy the assumption in Section 2, the weights must be determined such that wi
___ wi
Suppose that the pair-wise comparison matrix Q is available for the weighted secretary problem.
for i #j.
qij,
TO 21
3j 4)
;;
LP
= wj
S12 + 513 + S14 + S15 + S21 + S23 + S24 + S25 + S31 + S32 + 534 + 535 + 541 + S42 + S43 + S45 + S51 + S52 + S53 + S54
SUBJECT
END
+
9) 10) 11) 12) OPTIMUM
S12 - S21 + S13 - S31 + s14 - S41 + s15 - s51 + S23 - 532 + 524 - 542 + S25 - S52 + s34 - S43 + s35 - s53 + 545 - 554 + Wl + w2 + w3
0.75 Wl 0.67 Wl 0.75 wl 0.50 Wl 0.40 W2 0.50 W2 0.25 w2 0.60 w3 0.37 w3 0.25 W4 + w4 +
FOUND AT STEP
11
OBJECTIVE 1) VARIABLE s12 s13 s14 515 s21 S23 S24
0.25 0.33 0.25 0.50 0.60 0.50 0.75 0.40 0.66 0.25 =
0
W2 = W3 = W4 =
0 0
w5 = w3 = w4 = w5 = w4 = w5 = w5 = 10
.590000100
.oooooo .oooooo .oooooo .oooooo .oooooo
S25 531
.oooooo .010000
S32 s34 s35 s41 S42 s43 s45 551 S52
.oooooo .oooooo
s53 554 Wl
w5
FUNCTION VALUE
VALUE .oooooo .oooooo
.oooooo .oooooo .oooooo
.oooooo .oooooo .oooooo .oooooo
REDUCED COST 1.477333 2 .oooooo .oooooo
.562000 .522667 .968333 1.882000
.oooooo .oooooo 1.031667 1 .oooooo 2.000000 2.000000 .118000 1 .oooooo 2.000000 1.438000 2 .oooooo
.080000 .500000 1.000000
.oooooo .oooooo .oooooo
w2 w3
3.000000 2.000000
.oooooo .oooooo
w4 w5
3.000000 1.000000
.oooooo .oooooo
Fig. 3. LINDO
solution
(12)
If there is any inconsistency between the pairwise comparison matrix and the weights, it can be minimized via a goal programming approach
qij.
MIN
92 (1996) 135-147
to the goal programming
formulation.
Y.H. Chun /European
(Ignizio, 1976). With the deviational variables sij, the appropriate goal programming formulation would be min 2 = C sij
(13)
i#j
AHP, and Zahedi (1986) and Saaty (1990) for a discussion of applications of AHP. Example 4. Suppose a decision-maker’s
pairwise comparison matrix for five applicants is given as follows:
s.t. qjiwi - qijwj = sji - sij, w,+w,+
for all i fj,
**. +w,=10,
(14)
(15)
all variables nonnegative. The last constraint (15) is necessary to exclude the trivial solution, wi = 0 for all i. When the weighting scheme is not consistent with the pairwise comparison matrix Q, an element in Q can be expressed with an error term as follows: wi + Aij qii = ___ wi + w,
for i #j,
(16)
where -CC < A,, < + 03 and Aij + Aji = 0. Thus, in the goal programming formulation, it can be shown from (16) that the constraints in (14) is expressed as wj - Ajj wi + Aij
sji - sij = -w. wi+wj
- -w. I wi + wj
’
=~{(Wj-Aij)~,-(Wi+dij)Wj) I J
= Aji.
(17)
Thus, the objective function value in (13) is expressed as minz= =
Csjj=
C(sij+sji)
i#j
i
C i
(‘ji\T
145
Journal of Operational Research 92 (1996) 135-147
(18)
which implies that, if the weights obtained by the goal programming are perfectly consistent with the decision-maker’s pairwise comparison matrix, then the objective function value becomes zero. In that sense, the objective function value in the goal programming formulation can be interpreted as the consktency index as in Thomas Saaty’s analytic hierarchy process @HP). See the special issue of European Journal of Operational Research (Vol. 48, No. 1, 1990) for more details on
Q=
1 0.75
0.25 1
0.67 0.75 I 0.50
0.40 0.50 0.25
0.33 0.60 1
0.25 0.50
0.50 0.75
0.4
0.60 0.34
1
0.66 0.75
0.25
1
Based on the pair-wise comparison matrix, we formulated the problem as a goal programming and solved it using LINDO in Fig. 3. According to Fig. 3, the optimal solution is W= (1,3,2,3,1}, which has been used in the previous examples. In fact, the pairwise comparison matrix was derived arbitrarily from the weight W = (1,3,2,3,1}. However, due to the rounding-off error in the pair-wise comparison matrix, the objective function value or the consistency index is not zero in Fig. 3. 7. Concluding remarks For the weighted secretary problem, we have derived the optimal selection strategies under which the probability of selecting the best choice is maximized. We have also considered an ordering strategy for the case where the decision maker can rearrange the sequence of choices. Our results suggest several possible directions for further investigation. Some of these efforts, regarding the weighted secretary problem, are currently being developed and include consideration of (1) selecting more than one candidate, (2) using the maximum entropy criterion, (3) evaluating choices using multiple criteria, and (4) assuming that the number of applicants is not known in advance. The classical secretary problem and its extensions have been successfully applied to a wide variety of different situations arising in operational research applications. Since the weighted secretary problem is a more generalized version of the classical secretary problem, the weighted secretary problem may be applied to most areas where the classical secretary problem has been
YH. Chun /European
146
Journal of Operational Research 92 (1996) 135-147
successfully applied. Some of the possible applications are in consumer choice behavior (Nelson, 1970 and Rapoport and Tversky, 19701, job search (Lippman and McCall, 1976 and McCall, 19651, stock price prediction (Hlynka and Sheahan, 1988), sequential assignment of batched jobs (Albright, 1974; Albright and Derman, 1972; Derman et al., 1972), the house selling problem (Albright, 1977), and in searching for the lowest price product or service (Telser, 1973 and Rothschild, 1974).
which implies that 12
i
wk
k=j*+l
*
‘k-1
It also follows from (8) that q*-2
-l)=v
’
n =-
wk
c k=;*-1 vk-l
y*_,
K i Wj*-1
Appendix A. Proof of Lemma 1
--
K
By (61, this lemma states that, if rc(j) > r&j>, then r,( j - 1) > r&j - 1) for j = 2,3,. . . ,n. The hypothesis rrcTT,( j) > rsTTs( j> implies that r *(j) = r&j). Meanwhile, it follows from (2) with the value j reduced by one that v, r,(j),
which implies that
k
I
I
n
,$*+=
Y-1
rc(j-l)=$rr*(j)+-
wj*-l
=7r(ji)-7
which is rcTT,( j - 1) = r&j> because r *(j) = TG(j) from the hypothesis. Thus, the hypothesis r&j) > rs(j> implies that rc(j - 1) > r&j). Since rs(j> is increasing in j as shown in (11, we see that rs,( j) > rsTT,( j - 1). Thus, we conclude rcTT,( j - 1) > rJj - 1). ? ?
??
1
Appendix C. Proof of Theorem 2 Suppose that wi < wi+ 1 where i E B. Then, it follows from (8) that the winning probability in this sequence is
Appendix B. Proof of Theorem 1 Note that rr( j * ) = max(rr( j) for Vj E S,). It follows from (8) that
+
h wk
p(j*+l)=S+
n
k=j'+l
‘k-1 wi* \
If we interchange the two choices at stages i and i + 1, then the winning probability in this rearranged sequence becomes
Y.H. Chun /European
The difference
Journal of Operational Research 92 (1996) 135-147
in the winning probabilities wi
wi+1
K-1 +
--
K-1 + wi+1
Bellman, R., and Dreyfus, S. (19621, Applied Dynamic Programming, Princeton University Press, Princeton, N.J. Chow, Y.S., Moriguti, S., Robbins, H., and Samuels, SM. (19641, “Optimal selection based on relative rank (the ‘secretary problem’)“, Israel Journal of Mathematics 2,
is
wi
K-1
81-90.
wi+1
Chow, Y.S., Robbins, H., and Siegmund, D. (19711, Great Expectations: The Theory of Optimal Stopping, Houghton Mifflin, Boston. Clemen, R.T. (1991), Making Hard Decisions: An Introduction to Decision Analysis, PWS-Kent, Boston. Derman, C., Lieberman, G.J., and Ross, S.M. (19721, “A sequential stochastic assignment problem”, Management Science 18, 349-355. Dynkin, E.B. (1963), “The optimum choice of the instant for stopping a Markov process”, Soviet Mathematics Doklady
r/;-, + Wi =-4-l K
wi+l
wi
wi
K-1
+
W z+l
-
K-1
+wi+1 wi
w,
l/_,+wi
-
I/;._l+wi
147
1
4, 627-629.
Ferguson, T.S. (1989), “Who solved the secretary problem?“,
1
1
[
-Wi
y-1wi( r ~_1+Wi
wi+l
zz-
K
y-1(
-
If-1
-
5-1
=-
K
(K-1
+
wi(wi+l
+
-
Ii
wi)
Journal 61, 35-73.
Wj)
wi(wi+l -
I/;:-l+wi+l
wi)(V,-l
-
Hlynka, M., and Sheahan, J.N. (19881, “The secretary problem for a random walk”, Stochastic Processes and Their
wi) +
Applications 28, 317-325. wi+l)
wi)
(K-l+wi)
1 -x i K-1
Statistical Science 4, 282-296.
Freeman, P.R. (1983), “The secretary problem and its extensions: A review”, International Statistical Review 51, 189206. Gilbert, J.P., and Mosteller, F. (19661, “Recognizing the maximum of a sequence”, American Statistical Association
1 (K-1 +wwi+1) 1
Ignizio, J.P. (19761, Goal Programming and Extensions, Lexington Books, Lexington, MA. Lippman, S.A., and McCall, J.J. (19761, “The economics of job search: A survey”, Economic Inquiry 14, 155-189. McCall, J.J. (19651, “The economics of information and optimal stopping rules”, Journal of Business 38, 300-317. Nelson, P. (1970), “Information and consumer behavior”, Journal of Political Economy 78, 311-329.
-
Rapoport, A., and Tversky, A. (19701, “Choice behavior in an optimal stopping task”, Organizational Behavior and Hu-
which implies that we can increase the winning probability by rearranging the choices in B in non-increasing order of their weights. ? ?
Ross, S.M. (19701, Applied Probability Models with Optimization Applications, Holden-Day, San Francisco. Ross, SM. (19831, Introduction to Stochastic Dynamic Programming, Academic Press, San Diego. Rothschild, M. (1974), “Searching for the lowest price when the distribution of prices is unknown”, Journal of Political
=-“;.-I
v,
wi) Wifl (K-1 +wi) K-dvi-1 +?+I)
wi(wi+l
> 0,
man Performance
5, 105-120.
Economy 82, 689-711.
References Albright, SC. (19771, “A Bayesian approach to a generalized housing selling problem”, Management Science 24, 432440.
Albright, S.C. (1974), “Optimal sequential assignments with random arrival times”, Management Science 21, 60-67. Albright, S.C., and Derman, C. (19721, “Asymptotic optimal policies for the stochastic sequential assignment problem”, Management Science 18, 46-51.
Saaty, T. (1990), Multicriteria decision making: The analytic hierarchy process, RWS Publications, Pittsburgh, PA. Sakaguchi, M. (19841, “Bilateral sequential games related to the no-information secretary problem”, Mathematics Japonica 29, 961-973.
Schwertman, N.C., McCready, T.A., and Howard, L. (19911, “Probability models for the NCAA regional basketball tournaments”, American Statistician 45, 35-38. Telser, L.G. (19731, “Searching for the lowest price”, American Economic Review LXIII, 40-49. Zahedi, F. (19861, “The analytic hierarchy process - A survey of the method and its applications”, Interfaces 16, 96-108.