- Email: [email protected]
Search for a moving target when searcher motion is restricted
Cmpat. & 9s Rc& Vol. 6. pp. 129440 Pmgamon Press Ltd.. 1979. Printed in Great Britain
SEARCH FOR A MOVING TARGET WHEN SEARCHER MOTION IS RESTRICTED* T. J.
STEWART?
National ResearchInstitutefor MathematicalSciences, CSIR, PretoriaOool, Republicof South Africa
mf=d
ptqeee-In many cases involving the solution of search problems, resources camrot insexy be re-located,but searchersare constrainedto follow de&ritetracksat speeds which are slow comparedwith detection times. Gaul this aspect has been investigatedtheoretically,no correspond&t numerical algorithmshave been avaihrble,and the subject of this paper is thus the development of algorithmsfor completelyoptimizingsearchermotion. Abstract-A problemof search for a movingtargetin discrete space and time is consideredfor the case in which the rate of re-location of resources is constrained. Algorithmsare developed for numerically determiningsearch policies when resources are arbiiy divisible and when they are indivisible(single searcher).A potential heuristic approachto the problemof limited resourcedivisibilityis also discussed. The approachesare comparedby using as an example the search for a one-dimensional randomwalker. 1. INTRODUCTION
We consider the problem of allocating resources to the search for a single, possibly moving, target, but where such resources can be redeployed at a limited rate only (m contrast to radar sweeps for example, or in some cases aerial search), Such situations arise p~ic~~ly where search is carried out by physical manned patrols. These may consist of a single indivisible entity or may to a greater or lesser extent be divisible into sub-patrols. In this paper primary consideration is given to the cases where resources are either arbitrarily divisible or indivisible. Some comment is however, made on the intermediate case. The problem being considered contrasts with that dealt with in most work on search theory (see e.g. Stone[6]), where there may be constraints on total search effort and on rate of application of search effort; but where these constraints for any time t do not depend on the location of effort expended earlier than t. There has nevertheless been some discussion of constrained searcher motion. Lukka [3] treats this problem in continuous time, formulating it as an optimal control problem, the control variable being the velocity of the searcher. A single searcher (~divisible resource) only is assumed. His results are not, however, conve~ent for ~mputi~ controls except in certain simple cases, and an extension to more than one searcher would, owing to dimensionality problems, become considerably’less convenient (and impossible for arbitrary divisibility). Ciervo[2] also devotes attention to the problem we are considering. The main thrust of his work is the development of methods of computing detection probabilities for given searcher paths. His results can be used to compare previously specified search plans, or, by use of standard non-linear programming techniques, to optimize plans within a class defined by a small number of parameters. Complete optimization of searcher paths is not in general practicable with this approach. There is thus a need to investigate practical approaches to the compu~tion of optimal search plans under the conditions sketched above, and this is the purpose of the present paper. For practical reasons it is necessary to work in terms of discrete space and time descriptions. Under this restriction an algorithm can be developed for the case of infinitely divisible search effort and exponential detection function which is proved convergent to the globally optimal solution. In many cases an assumption. of in&rite divisibility will be quite satisfactory provided the this work was partiallysupportedby a grantfrom ControlData, YI’heoJ. Stewart is head of OperationsResearch and Statistics in the National Research Institutefor Mathematical Sciences of the Councilfor Seientigc and IndustrialResearch,Pretoria,South Africa. He holds degreesof B.Sc. (Chemical Engineering)KJniversityof Cape Town), Hons. B.Sc. KJperationsResearch), MSc. (OperationsResearch) and Ph.D. (MathematicalStatistics)(University of South Africa). He had experience ia ChemicalPlant managementbefore entering OperationsResearch. 129
R. WISK
130
T.J. STEWART
number of individual elements participating in the search (for example, men on patrol) is large enough. A completely different class of algorithm is required when the searcher effort is indivisible (i.e. a single searcher is constrained to move on a continuous path). A branch-and-Lund algorithm is developed in principle for the solution of this problem. For practical computational reasons, however, exact bounds cannot be obtained, and thus optimality cannot be guaranteed. Computational experienbe nevertheless indicates that the algorithm consistently produces optimal or very nearly optimal plans. The intermediate case, of limited divisib~ity but not indi~sib~ity of effort, is also discussed. When restrictions on divisib~ity are severe, the algorithm which was developed for i~nitely divisible effort does not behave well, even as a heuristic approximation. it is shown that in such cases it is preferable to develop a search plan in a sequential manner, deriving a search plan for each searcher in turn using the branch-and-bound algorithm for a single searcher, based on target location probabilities conditional on failure by those searchers for whom search plans have already been derived. Computational behaviour of the algorithms is compared by means of an example in which search is directed towards the detection of a one-dimensional random walker, a variation of the problem formulated by McCabe[4] (who did not obtain optimal solutions, but proved that solutions with bounded expected time to detection existed; existence of optimal solutions is shown in Stone[f?). This problem also has some interest in its own right.
2.FORMULATIONOFTHEPROBLEM Let us now formulate the problem which will be considered in this paper. We restrict ourselves to discrete space and time. The space is represented by a set of cells J, and the location of target and of any search effort is represented only by a member of J. Time is represented by a discrete number of periods denoted t, t = 1,. . . , T. Over a given period t both the search effort in each cell j E J and the location of the target are assumed to be fixed. The target is not assumed to be stationary in general. Thus we define a target path by the vector 0 = [o(l), w(2) ,. . . ,o(T)], where o(t) E f denotes the location of the target at time t. In general u is unknown, but we assume that a probability p. can Qpriori be associated with each sample path o. Search effort placed in j EJ during the period t is denoted by X(j, t). If o(t) = j, then the probability of detection during t, given that the target is not detected earlier, is assumed to be [l -exp {-&X(j, t)}], independent of X(i, T) i# j, t# T (where & is a type of sweep-width parameter, as in Stonef6J). The constraint on searcher motion is represented by sets I(j), one for each j f 1, where I(i) is the set of locations to which the search resources represented by X(j, t) can be re-located for time period t + 1. Thus if we define yfj, k, t + 1) as the effort re-located from j to k at the end of period t, then we have for 0 d t 6 T - 1:
This clearly implies that for all f
~je,XCi,t) = iv
(3)
where N represents total search resources available. Frequently (but not necessarily) N may be an integer number of searchers. In the above formulation the initial location of the resources, designated X(i, 0), is assumed known. We further allow that the amount of effort shifted between two cells from one period to
Search for a moving target when searcher motion is restricted
131
the next may be constrained, i.e. we include the following ~equ~ity: 06yyO’,k,t+l)~U(j,k)
kEI(j),
k#j
(4)
where U(j, k) is specified as an upper limit on relocation of resources from cell j to cell k within one time period. Finally the objective treated in this paper is that of maximizing probability of detection up to time T. We note that with the exception of the constraint on searcher motion, the problem is that for which Brown[l] derived an algorithmic solution. Some of the ideas of his paper have been incorporated here. 3. SOLUTION WITH INFINITE
DIVISIBILITY
OF RESOURCES
In this section we allow IV and X(j, t) to be any real rmmbers (i.e. not necessarily integraI multiples of some fixed value) satisfying the constraints. Our approach is based on the observation that the constraints (l)-(4) are typical networktype constraints (see Simmonard[5], chapters 13 and 14). To illustrate, assume J = (1, 2,.. . ,n} and define a network of n(T + 1)-t 1 nodes, the last labelled as a ‘sink’, and the remainder IabeIled (t,j) for t =O,l,..., T and j EJ. The constraints define feasible flows in the network, supplies at (0, j) for j E J of X(0, j), and demand at the sink of N. AII other nodes are transfer nodes only (no supply or demand). Arcs join node (t, j) to (t + 1, k) for all k E I(j), for each t = 0, l,..., (T - l), and flow on each of these arcs is y(j, k, t f 1). Each node (T, j) is joined to the sink by a sir&e arc, with associated flow X(i, 7’). The constraint (4) defines capacities for various arcs (and requires non-negative flow). The problem thus becomes one of choosing network flows, satisfying given supplies, demands and capacity constraints, and optimizing some criterion function. Clearly at this stage flows could be required to be either integers (implying a finite divisibility of resources) or real (infinite divisibility). The criterion function is unfortunately non-linear. The existence of a network-type formulation strongly suggests, however, that some form of linearization should be used, making it possible to employ standard linear network flow algorithms (see, e.g. Simmonard[lS], chapters 13 and 14). It wilI now be demonstrated that for infinitely divisible resources, a convergent optimi~tion agony can be constructed on this basis. For finitely divisible resources this is no longer true in general (in spite of the sui~b~ity of network algorithms for integer problems), although the method may serve as a suitable heuristic approach in some cases. The probability of no detection within 2’ periods is easily seen to be as follows (after conditioning on the true target path):
(6) where the func~on~ dependency of Q on the decision variables y(j, k, t) will, for notational conve~en~e, not be demonstrated explicitly. The objective is thus to select the flows y(i, k, t) in such a way as to minimize Q. We now note the following: Convexity property The problem (of minimizing Q with respect to the y(j, k, t)) is a convex programming
problem. This property is easily seen. The objective is a convex sum of convex terms. The constraints are linear and thus form a convex set.
132
T. J.
STEWART
Now consider a linearization 0 of (4) around a nominal set of flows F(-, *,+) with corresponding Q = Q The linearization can be written
(7) where P(k, t; 8) as defined in the above is the probability that the target is in cell k at time I and remains undetected t~ou~out the whole search defined by jr(-,*,e). For M~kovian searcher motion a useful algorithm for computing P(k, t; y’)is given by a suitable modification of the algorithm given by Brown[ I]. Based on the above linearization, we have the algorithm below, which, owing to the convexity property, is of the convex-simplextype (see Zangwill[7],Section 8.2),except that the network routine replaces the standard L.P. simplex. Zan8will[7]does prove convergence for this class of algorithm, provided steps (4) and (5) are replaced by a linear search for the minimum of Q along the line joining y(-, -, -) and y(*,a,m).We state the algorithm in the form below because approaches such as that in steps (4) and (5) have usually been found to be compu~tion~y more efficient than performing the full linear optimization step. ~~0~~ other non-linear pro~rn~ ~go~t~s may’also be used, the advantage of the one below is that it makes use of the fast network ~orit~s which are available as a result of the special structure of the problem.
4. ALGORITHM FOR INFINITELY
DIVISIBLE
RESOURCES
(1) Start with an arbitrary set of flows y(j, k, t) and some small real number E > 0. (2) Compute Q and P(k, t; 8) from (6) and (7). (The computation of Q for Markovian motion is also facilitated by Brown’s algorithm.) (3) Solve the linear network problem, with network constraints as defined above, aud with costs [-p&k, t; Y)] associated with flows y(i, k, t); zero costs are associated with the flows from the nodes (i, T) into the sink; let the value of Q ~~es~nd~g to the solution to the linear ~~~~tion probiem y(j, k, 1) be denoted Q and define: A = M=%j>,]Yt.j,k, t) - Yci,k, 01. (4) If Q < Q, go to (6), otherwise go to (5). (5) If A< l, go to (7). Otherwise replace y(j, k, t) by {y(j,k, t) t ffj, k, t)}/2 and set A = A/2. Recompute Q and return to (4). (6) Replace j%j,k, t) by y(j, k, t) and return to (2). (7) Stop, with search plan defined by f(-, *,-) through constraint (2). When resources are not arbitrarily divisible, the problem arises that the constraint set is no longer convex. It is thus no longer necessarily true that the algorithm converges to the optimal solution. (Step (5) must also be mod~ed in order to force feasible ~te~tives.~~re a subs~ti~ (but still not arbitrary) degree of divisibility is allowed, the algorithm may still be expected to perform well (with appropriate modification of step (5)). Performance with more limited divisibility (say iV < 10, and y(i, k, t) integers) is not at all evident, and this issue is pursued further in subsequent sections. Thus we have a useful algorithm when the rate of transfer of resources is limited, but divisibilityof these is for practical purposes not limited; This is a useful result when applicabIe, although in most practical situations of the type considered here the divisibility of resources is severely limited. For this reason our computational results below are restricted to such cases.
Search for a moving target when searcher motion is restricted 4. INDIVISIBILITY
OR LIMITED DIVISIBILITY
133
OF RESOURCES
From now on we assume that iV is an integer (usually a number of searchers) and that XQ t) and y(i, k, t) are constrained to be integers. As indicated at the end of the previous section, we may be tempted to use the algorithm of that section in this case as well, even for N quite small. Note that for small N(N,$5, say) the difficulty with step (5) for integer solutions can be avoided by dropping the step entirely and replacing constraint (4) by: Max [O,jQ, k, t) - 11d y(i, k, t) 5 Mi [ UCj, k), y(j, k, t) + 11.
(8)
(For larger N an unnecessarily large number of network flow problems may have to be solved if this approach is used.) In the next section it is demonstrated by means of an example that at least in the cases treated here (N s 4), this algorithm cannot be relied upon to yield even near-optimal results. Thus some other approach is clearly needed for small N. We therefore now switch attention to the other extreme, viz. N = 1 (i.e. indivisible resource, requiring the constraint X(i, t) E (0, I}). This reduces the problem to a (0,l) integer (non-linear) programming problem. Although a solution in terms of a dynamic programming approach can in principle be written down, the problem of dimensionality effectively precludes the use of such a solution in practice. State description at any stage requires specification of posterior probabilities for the position of the target (one for each element of J) as well as specification of the current searcher position. Most current approaches to linear (0, I) problems are of the branch-and-bound (implicit enumeration) type. Thus we were led to attempt such an approach here. The basic outline of the branch-and-bound procedure is quite simple and is given below. In the description we say that if an arc from node (t, j) to the node (t + 1, k), k E Z(j) (still using the network formulation) has either been eliminated as being non-optimal, or is included in a solution that has been explicitly evaluated, then it is ‘fathomed’. Associated with this concept we define the set K(t, j) as {klk E Z(j) and the arc from (t, j) to (t + 1, k) is unfathomed} for each t, j. We assume again that J={l,2 ,...) n}. Outline of branch-and-bound algorithm (1) Set t = 0, p = 1; let j0 E .Zbe the unique index such that Xf&, 0) = 1; set K(0, jO)= I&). (2) Solve the substitute problem (discussed below) for search over periods t + 1, t + 2,. . . , T, with searcher at jt prior to the search, and with searcher location in the tirst period (i.e. t + 1) restricted to K(t, j,). Obtain thereby (also discussed later) a lower bound p on the optimal probability of no detection when searcher paths are restricted to those passing through jO, JI,...,ZP (3) If P 0, go to step (5). (5) Delete the current j, from K(t - 1, j,-J and set t = t - 1. If K(t, j,) is now empty, the arc from (t - 1, a_,) to (t, j,) is implicitly also fathomed. Thus return to step (4). Otherwise return to step (2). (6) Select the element of K(t, jt) appearing in the solution to the substitute problem and call this j,+,; set t = t + 1. If t < T, set K(t, j,) = Z(J) and return to step (2). If t = I”, evaluate the probability of no detection when the searcher path is jO,jr , . . . , jr. If this probability is less than @,replace P by this value and instal jO,ji ,... ,jT as the current best solution. Return to step (5). Very evidently the crucial question is the selection of the substitute problem and utilization of its solution in inferring the bound P and in selecting the new jr+r. Two approaches suggest themselves. The first turns out to be not very suitable for the branch-and-bound procedure as such, but is described because it does display some interesting properties. For both approaches the substitute problem is a search problem, the initial target location probabilities of which are those conditional on no detection up to time t on the current path (i.e.
T.3. STJYMKF
134
jo,il,
* - - jr). The probability of no detection on optimal continuation of the path is given by the product of 9
Prlno detection up to t on path jotjt , . . . , jt] and Pr[no detection on the optimal continuation jt+r,. . . , jTI no detection on path jo,jr,. . . , jJ. The first term can be computed from jo, jI , . . . , j t (and is unity for f = 0). Thus the lower bound on probability is obtained from a lower bound on the second term only, the associated search problem beii that of the substitute problem. For this reason we need discuss the substitute problems and their ~s~iated botmds for the case of t = 0 only. The other eases follow directly from the preceeding remarks. (Clearly the bench-and-Lund ~go~t~ must then also provide for updating of relevant probabiities.) The first approach is suggested directly by the network form of constraint. The true objective may be viewed as minimizationof expected probability of no detection conditional on w, which is a re-interpretation of (6), written as: Q = x@p,[pr{uo det~~onl~get
path = o}].
Let us now rather consider rn~i~tiou of the expectation of the logarithm of this conditional probab~ity. The objective thus becomes mi~mi~tion of:
where
is the probability that the target path passes through cell k at time t. This is now a linear objective function, and together with the network constraints forms a standard linear network flow problem which can be solved directly. A bound may be obtained from Jensen’s inequality. To see this, let us express equation (6) in the following simpiifi~ form:
(where no,[y(*,a,+)Iis the condition probab~ity of detection when the target path is w and searcher motion is defined by y(*,0,a)).Then let y*(*,e,-) represent optimal searcher motion and let y’(1, ) represent searcher motion when the above substitute problem is being solved. We thus have log Q* L x,pw
log ~~[y*(*,1,a)](by Jensen’s in~qu~i~) (9)
where the second inequality follows by optima&y in the substitute problem. ~~o~unately it appears that the above ineq~i~ is too weak to be usefully effective in the branch-and-Lund procedure. We have included discussion of this approach here partly to warn
Search for a movingtargetwhen searchermotion is restricted
I35
off potential users, but also because of the interesting property noted in the next section, viz that the first full solution generated is usefully near-optimal. This suggests a potential heuristic procedure for large problems (i.e. following the branch-and-bound procedure only until t = T for the first time). The practicality of such a procedure depends heavily, however, on the efficiency of the network algorithm used, and although we have not as yet been able to implement the suggestion in any computationally attractive way, it was considered sufficiently interesting to warrant mention. The second approach is to use as substitute problem in the branch-and-bound procedure a search problem with relaxed constraints, i.e. in step (2) of the branch-and-Lund ~gori~rn we solve a search problem with moving target, with constraints on resource availability per period, but not on searcher motion as such. Essentially, this probelm requires choice of one cell each period (since we require X(i, t) E (0, 1}), where choice in period T (t + 1 d r G T) is limited to that reachable from j,, i.e. choice must be made from a set 1, where:
&+I= KO, iA J, = ulj E I(k) for some k E .LJ. It is not required here, however, that j7 should be a member of Ii’jl-l). If the resource indivisi~ity constraint is also relaxed (i.e. if we allow 0 C X(i, t) 6 1 rather than X(‘j, t) E (0,1)) then a solution can be obtained nu~ri~aIly by using Brown’s[ If algorithm. Experience has, however, shown that this further relaxation results in bounds which are also too weak for practical use. Brown’s algorithm can still be used while the indivisibility constraint is retained, but owing to the non-convexity of the feasible region defined by the indivisibility constraint, the solution obtained satisfies only necessary (but not sufficient) conditions for optimality. Thus optimality in the substitute problem cannot be guaranteed, and the implied bound on the original problem may be incorrect (which may result in the elimination of an optimal continuation). As in other non-convex problems, the chances of identifying the true optimal solution can be improved by repeating Brown’s algorithm for a number of different initial plans. Our experience is, however, that this approach yields little or no practical improvement. In fact, by using Brown’s agony for the substitute problem with the ‘myopic’ initial plan as suggested in [I], we have achieved optimality or very near-optim~ity with the branch-ad-Lund approach in ah of a large number of test cases. In over 100 test cases in which the true optimal solution was known, this solution was found in all except 4 of the cases, and in the remaining cases the probability of no detection never exceeded the minimum probability by more than 0.41% of its value. For these reasons we propose use of Brown’s algorithm for the substitute problem with retention of the indivisibility constraint, although it must be conceded that this gives a good heuristic rather than strictly optimal solution. Retention of indivisibility in fact simplifies Brown’s algorithm substantially. For the sake of completeness, the steps of this simplified dlgorithm are outlined below in the nomenclature of the present paper. (Recall that in the ~gorithm the target path proba~iIities are conditions on failure of the search over the searcher path jet, jr ,. . . ,j*; again for Markov motion these path probabilities are easily generated recursively.) Brown’s scheme for the relaxed problem (1) Start with an arbitrary set of search locations i(T) E 1, for f + I S T 5 T. Select a suitable small positive number E. Set B = 1 and T = t + 1, and go to step (2). (2) Evaluate for each k E Jf:
where S(i, j) is the usual Kronecker delta. CAOR
Vol. 6 No. 3-S
T. J. STEWART
136
(3) Replace i(7) by j satisfying
(I- exP (-&)HYj) = Maxkl{l - exp (-#3&P(k)]. If r c ‘I’,set T = 7 f 1 and return to step (2). Otherwise proceed to step (4). (4) Compute 6 = kz P(k) +PMT)l =P
[-i%(T)&
kM *F f
necessity 6 S @.If P - 6 < E, then stop: within au accuracy of E, f$ is the lower bound on probab~ity of no detection on connation; f~he~ore we may set jr+,= i(f + 1) in the blah-Ed-Lund procedure. If P - 0 I 6, then set @= 0, T = f + 1 and return to step (2). As shown in the next section, this approach is very suitable for the branch-and-bound procedure; in some examples this approach has reduced by a factor of about ld the total near of branches to be evaluated, and by a factor of about 108the number of solutions explicitly enumerated. This yields a practical solution of the single-searcher(indivisibleresource) case. In the case of multiple searchers, the number of branches required in the corresponding branch-and-bound procedure increases ex~nenti~y with N, and this approach would soon cease to be viable. When N is too small for the infinitely divisible algorithm to be used satisfactorily, a convenient heuristic approach is to allocate searchers sequentially, using the single-searcherbranch-andbound procedure for the allocation of each in turn, but with p~bab~ties taken conditional on no detection by the previously allocated searchers. We cannot make any rigorous statement on closeness to optimahty of this heuristic. Nevertheless, apart from its intuitive appeal, such a ‘greedy’ approach (in which the greatest marginal value is obtained at each successive notation) has some jus~~tion here for N small in ~omp~son with the number of cells to be searched, since in such cases there is little overlap of effort between searchers.
Of
S.ACOMPUTATIONALEXAMPLE
As has been indicated, we restrict attention here to those cases where there is limited divisibility of resources. A vehicle particularly well-suited for comparing the various approaches is a variation of the problem suggested by McCabef4],viz. the problem of search for a one~~ensio~ random walker. The example is of some interest in itself in that it is a one-diiensional analogue of a typical counter-ins~gency search problem in which the search patrol would first proceed to the scene of some event and thereafter spread out, in pursuit of the insurgents, whose movements are so unpredictable a priori as to be effectively a random walk from the point of view of the searcher. The solution to the on~~ension~ case can suggest the rate at which search should spread out from the origin, even in more general cases. Thus we consider a target whose position at time f, t L -r for some given r > 0, is given by xr, where: xwr=O Pr(xt+,--x*= l)=Pr(X~+,-x~ =-I)=;. In order to keep the model simple we shall assume that r is an even number. This will imply that crossings of the target path and a searcher path only occur if and when both searcher and searched arrive at the same location at the same time. Only at such crossings is detection possible. At this point we depart from McCabe’smodel in two respects. Firstly we allow N searcher paths, each of which satisfies the following for i = 1,. . . ,N: 441
E(-191)
]#$$*I- Cpi,lS 1 for f2 1
Search for a moving target when searcher motion is restricted
137
where & is the position of searcher i at time t. (Search thus starts at t = 1, with all searchers
orighally at the origin.) Secondly, instead of certain detection when paths cross (requiring & = 00for all j in our model), we assume a certain overlook probability a, i.e. & = -In a for all j. We compare the behaviours of the various approaches as they are applied to this problem, seeking to maximize probability of detection within time T. For computational efficiency and for ease of applying the results of the previous sections, it is preferable at each time t to work in terms of t + r + 1 ‘cells’, where cell j corresponds to position [-(t + r)+ 20’- l)] on the random walk. The target path probabilities pU can be derived from: (1) probability that the target is in cell j at t = 1, which is r+l
(I’:)(:) .
-
;
(2) Markov transition probabilities: Pr[target is in cell j at time t + 11target is in cell i at time t] =iforj=i,i+l = 0 otherwise. The searcher constraints of Section 2 are defined by: (1) X((r/2)+l,O)=N =0 Xti, 0) (2) Z(j) = b, j + 1).
j#(r/2)+1;
We consider firstly use of the method of Section 3, modified as suggested at the beginning of Section 4 (i.e. the network form of the convex-simplex method, used in spite of the limited divisibility of resources). Not surprisingly, little success was had for N = 1, but this situation is in any case of little interest here, as the branch-and-bound procedure yields a useful solution. For purposes of comparison the cases N = 2 and N = 4 were considered for time horizons T = 10 and T = 20. Results with overlook probability a = 0.2 are shown in Table 1 for three different starting values for searcher paths. For N/2 of the searchers the following were the three starting values (the others being mirror images of these): starting path (a): 0, 1, 0, -1, 0,. . . starting path (b): 0, 1, 2, 3,4,. . . starting path (c): 0, 1,0,- 1, 0, 1,2, 1,. . . (with change of direction whenever l&l is a Fibonacci number-a suggestion by McCabe). Table 1. Probabilities of no detection for multiplesearcherswith plansselectedaccordingto daerent schemes
t N-2 Scheme
Modified infinitedivisibility algorithm
Sequential use of single-searcher branch-and-bound algorithm
initial path
N=4
1 = 10
T = 20
T = 10
T = 20
(a)
0.09471
0.06403
0.02051
0.01229
(b)
0.09228
0.07484
0.01556
0.00920
(cl
0.09391
0.08456
o.a2051
0.00545
-
0.06537
0.01878
0.00691
0.00232
T. J.
138
STEWART
The results in Table 1 are compared with those obtained by sequentially allocating searchers by the branch-and-bound procedure, as suggested at the end of Section 4. Two conclusions are evident. Firstly with these numbers of searchers the modified infinite-divisibility algorithm performs considerably less well than the sequential branch-and-bound procedure. Secondly performance of the algorithm is strongly dependent on the starting solution, but no single starting solution is consistently best. If thus appears that for N of these magnitudes this algorithm is of little use. Computer running times were, however, less than the computer time required for the branch-and-bound approach, and appeared to be but little influenced by the value of N. In the branch-ad-bond procedures the total number of branches to be evaluated (i.e. the m~imum number of substitute problems to be solved) is (2 + 4 -t * - * + ZT-‘) = 2(2’-’ - l), while the total number of possible solutions is 2’. The efficiencies of the two potential approaches to the branch-and-bound procedure can be compared in terms of the numbers of each of these problems which have actually to be evaluated. In using the first approach and the bound in (9), it was found that for T = 10 the total number of substitute problems to be solved was 848 (out of a maximum of 1022), and 594 (out of 1024) full solutions had to be enumerated. Even when an approximate bound was used, based on a second-order Taylor expansion for log T_, for which expectations were taken (which adds a variance term to the right-hand side of (9)) this did not improve the situation substantially. Clearly it is about as ~ompu~tion~ly efficient to enumerate and evaluate all possible searcher paths. This first approach is thus of no value as a branch-and-Lund procedure, but a most interesting property is illustrated in Table 2. This table gives for a single searcher, for T = 10 and T = 20, and for overlook probability of 0.2, the probability of no detection on the optimal searcher path and on the first paths selected by each of two branch-and-bound approaches (the second discussed below). It is seen that the first selection with the first approach is considerably better than the first selection with the second. approach. What is more important, the result of the former approach compares well with the solution obtained by the branch-and-bound algorithm. It is this fact which suggests that this first selection may form the basis of a heuristic solution procedure which may be useful in some cases. Practical implementation of the procedure is very strongly dependent on the av~abili~ of a very efficient algorithm for the network flow problem (probably an ~go~t~ in which use is made of special s~ctures}. A standard genes-pose out-of-kiher algorithm proved to be not very efficient, and its use led to computation times for obtaining the tirst selection which were higher than those for obtaining the final solution by the second branch-and-bound approach. This difficulty has not yet been resolved. The second approach to the branch-and-bound procedure, based on using the relaxed problems for bounding and for branch selection, was found to be considerably more efficient. For T = 10 it required 105 (out of 1022) substitute problems to be solved and only 8 (out of 1024) solutions to be fully enumerated. For larger T the reduction was far more dramatic. The numbers required for T = 20 were 430 substitute problems and 13 full solutions (compared with 2M)and for T = 30 these were 1276 and 16 (compared with 2’“). This is a feasible approach to the solution of the problem of a single searcher with cons~ained motion.
Table2. Probabilities of no detection for a sin&e searcher For first path generated procedure
in the branch-and-bound Optimal
T 1st substitute problem (minimizing Eilog probability])
10 20
-
2nd substitute problem (relaxed constraints)
Solution
0.2348
0.2827
0.20740
0.1284
0.1858
0.08514
Search for a moving target when searcher motion is restricted
Fig. 1. Optimal searcher paths for T = 20and T = 30 and for overlook probability = 0.2.
Fig. 2. Optimal searcher paths for T = 20 and for overlook probabilities a = 0.05,O.u) and 0.50.
The compu~tion~ times on the CDC CYSRR 174 system were about 40 set for 2’ = 20 and 140 set for 7’ = 30. (The optimal solutions were in fact found within one-third to oneq~er of these times, the remaining time being required to confirm optimality). Finally in view of the intrinsic interest in this example, some solutions obtained with the branch-and-bound algorithm are displayed in Figs. I and 2. In Fig. 1 are shown the paths for T = 20 and 30 and overlook probability 0.2. It appears that a uniformly optimal solution does not exist, as these paths depend on the time horizon. In Fig. 2 the paths for three different overlook probabilities (a = 0.5, 0.2 and 0.05) are shown for T = 20. As could have been anticipated, the rate of expansion of search effort is slower for the higher overlook probabilities (when no detection is less conclusive). 6. CONCLUSIONS has been shown that the problem of opts search with constrained searcher motion can be solved numeric~y in the discrete time and space situation for infinitely divisible resources. It
140
T. J.
STEWART
When resources are indivisible, the heuristic branch-and-bound procedure provides a means of computing good plans in the form of a searcher track, although optimality cannot be guaranteed or proved in general. The intermediate case has not been investigated in depth, but a suggestion has been made as to how such problems may be handled as sequences of indivisible resource problems.
REFERENCES 1. Scott S. Brown, Optimal search for a moving target in discrete space and time. Tech. Rep., Daniel H. Wagner, Associates, Paoli, Pennsylvania (1977).(Submitted for publication.) 2. Anthony P. Ciervo, Search for moving targets. Pacific-Sierra Research Corp., Santa Monica, California, Rep. 619A (1976). 3. Markku Lukka, On the optimal searching tracks for a moving target. SIAMJ. Appl. Math. 32, 126-132(1977). 4. Bernard J. McCabe, Searching for a one-dimensional random walker. J. Appl. Pd. 11,&G93 (1974). 5. Michel Simmonard, Linear Programming.Prentice-Hall, Engkwood Cliffs, N.J. (1966). 6. Lawrence D. Stone, Theory of Optimal !&wch. Academic Press. New York & London (1975). 7. Willard 1. Zangwill, Nonlinear Progmmming: A UnijiedApproach. Prentice-Hall, Engiewood Cliffs, N.J. (PM).
SEARCH FOR A MOVING TARGET WHEN SEARCHER MOTION IS RESTRICTED* T. J.
STEWART?
National ResearchInstitutefor MathematicalSciences, CSIR, PretoriaOool, Republicof South Africa
mf=d
ptqeee-In many cases involving the solution of search problems, resources camrot insexy be re-located,but searchersare constrainedto follow de&ritetracksat speeds which are slow comparedwith detection times. Gaul this aspect has been investigatedtheoretically,no correspond&t numerical algorithmshave been avaihrble,and the subject of this paper is thus the development of algorithmsfor completelyoptimizingsearchermotion. Abstract-A problemof search for a movingtargetin discrete space and time is consideredfor the case in which the rate of re-location of resources is constrained. Algorithmsare developed for numerically determiningsearch policies when resources are arbiiy divisible and when they are indivisible(single searcher).A potential heuristic approachto the problemof limited resourcedivisibilityis also discussed. The approachesare comparedby using as an example the search for a one-dimensional randomwalker. 1. INTRODUCTION
We consider the problem of allocating resources to the search for a single, possibly moving, target, but where such resources can be redeployed at a limited rate only (m contrast to radar sweeps for example, or in some cases aerial search), Such situations arise p~ic~~ly where search is carried out by physical manned patrols. These may consist of a single indivisible entity or may to a greater or lesser extent be divisible into sub-patrols. In this paper primary consideration is given to the cases where resources are either arbitrarily divisible or indivisible. Some comment is however, made on the intermediate case. The problem being considered contrasts with that dealt with in most work on search theory (see e.g. Stone[6]), where there may be constraints on total search effort and on rate of application of search effort; but where these constraints for any time t do not depend on the location of effort expended earlier than t. There has nevertheless been some discussion of constrained searcher motion. Lukka [3] treats this problem in continuous time, formulating it as an optimal control problem, the control variable being the velocity of the searcher. A single searcher (~divisible resource) only is assumed. His results are not, however, conve~ent for ~mputi~ controls except in certain simple cases, and an extension to more than one searcher would, owing to dimensionality problems, become considerably’less convenient (and impossible for arbitrary divisibility). Ciervo[2] also devotes attention to the problem we are considering. The main thrust of his work is the development of methods of computing detection probabilities for given searcher paths. His results can be used to compare previously specified search plans, or, by use of standard non-linear programming techniques, to optimize plans within a class defined by a small number of parameters. Complete optimization of searcher paths is not in general practicable with this approach. There is thus a need to investigate practical approaches to the compu~tion of optimal search plans under the conditions sketched above, and this is the purpose of the present paper. For practical reasons it is necessary to work in terms of discrete space and time descriptions. Under this restriction an algorithm can be developed for the case of infinitely divisible search effort and exponential detection function which is proved convergent to the globally optimal solution. In many cases an assumption. of in&rite divisibility will be quite satisfactory provided the this work was partiallysupportedby a grantfrom ControlData, YI’heoJ. Stewart is head of OperationsResearch and Statistics in the National Research Institutefor Mathematical Sciences of the Councilfor Seientigc and IndustrialResearch,Pretoria,South Africa. He holds degreesof B.Sc. (Chemical Engineering)KJniversityof Cape Town), Hons. B.Sc. KJperationsResearch), MSc. (OperationsResearch) and Ph.D. (MathematicalStatistics)(University of South Africa). He had experience ia ChemicalPlant managementbefore entering OperationsResearch. 129
R. WISK
130
T.J. STEWART
number of individual elements participating in the search (for example, men on patrol) is large enough. A completely different class of algorithm is required when the searcher effort is indivisible (i.e. a single searcher is constrained to move on a continuous path). A branch-and-Lund algorithm is developed in principle for the solution of this problem. For practical computational reasons, however, exact bounds cannot be obtained, and thus optimality cannot be guaranteed. Computational experienbe nevertheless indicates that the algorithm consistently produces optimal or very nearly optimal plans. The intermediate case, of limited divisib~ity but not indi~sib~ity of effort, is also discussed. When restrictions on divisib~ity are severe, the algorithm which was developed for i~nitely divisible effort does not behave well, even as a heuristic approximation. it is shown that in such cases it is preferable to develop a search plan in a sequential manner, deriving a search plan for each searcher in turn using the branch-and-bound algorithm for a single searcher, based on target location probabilities conditional on failure by those searchers for whom search plans have already been derived. Computational behaviour of the algorithms is compared by means of an example in which search is directed towards the detection of a one-dimensional random walker, a variation of the problem formulated by McCabe[4] (who did not obtain optimal solutions, but proved that solutions with bounded expected time to detection existed; existence of optimal solutions is shown in Stone[f?). This problem also has some interest in its own right.
2.FORMULATIONOFTHEPROBLEM Let us now formulate the problem which will be considered in this paper. We restrict ourselves to discrete space and time. The space is represented by a set of cells J, and the location of target and of any search effort is represented only by a member of J. Time is represented by a discrete number of periods denoted t, t = 1,. . . , T. Over a given period t both the search effort in each cell j E J and the location of the target are assumed to be fixed. The target is not assumed to be stationary in general. Thus we define a target path by the vector 0 = [o(l), w(2) ,. . . ,o(T)], where o(t) E f denotes the location of the target at time t. In general u is unknown, but we assume that a probability p. can Qpriori be associated with each sample path o. Search effort placed in j EJ during the period t is denoted by X(j, t). If o(t) = j, then the probability of detection during t, given that the target is not detected earlier, is assumed to be [l -exp {-&X(j, t)}], independent of X(i, T) i# j, t# T (where & is a type of sweep-width parameter, as in Stonef6J). The constraint on searcher motion is represented by sets I(j), one for each j f 1, where I(i) is the set of locations to which the search resources represented by X(j, t) can be re-located for time period t + 1. Thus if we define yfj, k, t + 1) as the effort re-located from j to k at the end of period t, then we have for 0 d t 6 T - 1:
This clearly implies that for all f
~je,XCi,t) = iv
(3)
where N represents total search resources available. Frequently (but not necessarily) N may be an integer number of searchers. In the above formulation the initial location of the resources, designated X(i, 0), is assumed known. We further allow that the amount of effort shifted between two cells from one period to
Search for a moving target when searcher motion is restricted
131
the next may be constrained, i.e. we include the following ~equ~ity: 06yyO’,k,t+l)~U(j,k)
kEI(j),
k#j
(4)
where U(j, k) is specified as an upper limit on relocation of resources from cell j to cell k within one time period. Finally the objective treated in this paper is that of maximizing probability of detection up to time T. We note that with the exception of the constraint on searcher motion, the problem is that for which Brown[l] derived an algorithmic solution. Some of the ideas of his paper have been incorporated here. 3. SOLUTION WITH INFINITE
DIVISIBILITY
OF RESOURCES
In this section we allow IV and X(j, t) to be any real rmmbers (i.e. not necessarily integraI multiples of some fixed value) satisfying the constraints. Our approach is based on the observation that the constraints (l)-(4) are typical networktype constraints (see Simmonard[5], chapters 13 and 14). To illustrate, assume J = (1, 2,.. . ,n} and define a network of n(T + 1)-t 1 nodes, the last labelled as a ‘sink’, and the remainder IabeIled (t,j) for t =O,l,..., T and j EJ. The constraints define feasible flows in the network, supplies at (0, j) for j E J of X(0, j), and demand at the sink of N. AII other nodes are transfer nodes only (no supply or demand). Arcs join node (t, j) to (t + 1, k) for all k E I(j), for each t = 0, l,..., (T - l), and flow on each of these arcs is y(j, k, t f 1). Each node (T, j) is joined to the sink by a sir&e arc, with associated flow X(i, 7’). The constraint (4) defines capacities for various arcs (and requires non-negative flow). The problem thus becomes one of choosing network flows, satisfying given supplies, demands and capacity constraints, and optimizing some criterion function. Clearly at this stage flows could be required to be either integers (implying a finite divisibility of resources) or real (infinite divisibility). The criterion function is unfortunately non-linear. The existence of a network-type formulation strongly suggests, however, that some form of linearization should be used, making it possible to employ standard linear network flow algorithms (see, e.g. Simmonard[lS], chapters 13 and 14). It wilI now be demonstrated that for infinitely divisible resources, a convergent optimi~tion agony can be constructed on this basis. For finitely divisible resources this is no longer true in general (in spite of the sui~b~ity of network algorithms for integer problems), although the method may serve as a suitable heuristic approach in some cases. The probability of no detection within 2’ periods is easily seen to be as follows (after conditioning on the true target path):
(6) where the func~on~ dependency of Q on the decision variables y(j, k, t) will, for notational conve~en~e, not be demonstrated explicitly. The objective is thus to select the flows y(i, k, t) in such a way as to minimize Q. We now note the following: Convexity property The problem (of minimizing Q with respect to the y(j, k, t)) is a convex programming
problem. This property is easily seen. The objective is a convex sum of convex terms. The constraints are linear and thus form a convex set.
132
T. J.
STEWART
Now consider a linearization 0 of (4) around a nominal set of flows F(-, *,+) with corresponding Q = Q The linearization can be written
(7) where P(k, t; 8) as defined in the above is the probability that the target is in cell k at time I and remains undetected t~ou~out the whole search defined by jr(-,*,e). For M~kovian searcher motion a useful algorithm for computing P(k, t; y’)is given by a suitable modification of the algorithm given by Brown[ I]. Based on the above linearization, we have the algorithm below, which, owing to the convexity property, is of the convex-simplextype (see Zangwill[7],Section 8.2),except that the network routine replaces the standard L.P. simplex. Zan8will[7]does prove convergence for this class of algorithm, provided steps (4) and (5) are replaced by a linear search for the minimum of Q along the line joining y(-, -, -) and y(*,a,m).We state the algorithm in the form below because approaches such as that in steps (4) and (5) have usually been found to be compu~tion~y more efficient than performing the full linear optimization step. ~~0~~ other non-linear pro~rn~ ~go~t~s may’also be used, the advantage of the one below is that it makes use of the fast network ~orit~s which are available as a result of the special structure of the problem.
4. ALGORITHM FOR INFINITELY
DIVISIBLE
RESOURCES
(1) Start with an arbitrary set of flows y(j, k, t) and some small real number E > 0. (2) Compute Q and P(k, t; 8) from (6) and (7). (The computation of Q for Markovian motion is also facilitated by Brown’s algorithm.) (3) Solve the linear network problem, with network constraints as defined above, aud with costs [-p&k, t; Y)] associated with flows y(i, k, t); zero costs are associated with the flows from the nodes (i, T) into the sink; let the value of Q ~~es~nd~g to the solution to the linear ~~~~tion probiem y(j, k, 1) be denoted Q and define: A = M=%j>,]Yt.j,k, t) - Yci,k, 01. (4) If Q < Q, go to (6), otherwise go to (5). (5) If A< l, go to (7). Otherwise replace y(j, k, t) by {y(j,k, t) t ffj, k, t)}/2 and set A = A/2. Recompute Q and return to (4). (6) Replace j%j,k, t) by y(j, k, t) and return to (2). (7) Stop, with search plan defined by f(-, *,-) through constraint (2). When resources are not arbitrarily divisible, the problem arises that the constraint set is no longer convex. It is thus no longer necessarily true that the algorithm converges to the optimal solution. (Step (5) must also be mod~ed in order to force feasible ~te~tives.~~re a subs~ti~ (but still not arbitrary) degree of divisibility is allowed, the algorithm may still be expected to perform well (with appropriate modification of step (5)). Performance with more limited divisibility (say iV < 10, and y(i, k, t) integers) is not at all evident, and this issue is pursued further in subsequent sections. Thus we have a useful algorithm when the rate of transfer of resources is limited, but divisibilityof these is for practical purposes not limited; This is a useful result when applicabIe, although in most practical situations of the type considered here the divisibility of resources is severely limited. For this reason our computational results below are restricted to such cases.
Search for a moving target when searcher motion is restricted 4. INDIVISIBILITY
OR LIMITED DIVISIBILITY
133
OF RESOURCES
From now on we assume that iV is an integer (usually a number of searchers) and that XQ t) and y(i, k, t) are constrained to be integers. As indicated at the end of the previous section, we may be tempted to use the algorithm of that section in this case as well, even for N quite small. Note that for small N(N,$5, say) the difficulty with step (5) for integer solutions can be avoided by dropping the step entirely and replacing constraint (4) by: Max [O,jQ, k, t) - 11d y(i, k, t) 5 Mi [ UCj, k), y(j, k, t) + 11.
(8)
(For larger N an unnecessarily large number of network flow problems may have to be solved if this approach is used.) In the next section it is demonstrated by means of an example that at least in the cases treated here (N s 4), this algorithm cannot be relied upon to yield even near-optimal results. Thus some other approach is clearly needed for small N. We therefore now switch attention to the other extreme, viz. N = 1 (i.e. indivisible resource, requiring the constraint X(i, t) E (0, I}). This reduces the problem to a (0,l) integer (non-linear) programming problem. Although a solution in terms of a dynamic programming approach can in principle be written down, the problem of dimensionality effectively precludes the use of such a solution in practice. State description at any stage requires specification of posterior probabilities for the position of the target (one for each element of J) as well as specification of the current searcher position. Most current approaches to linear (0, I) problems are of the branch-and-bound (implicit enumeration) type. Thus we were led to attempt such an approach here. The basic outline of the branch-and-bound procedure is quite simple and is given below. In the description we say that if an arc from node (t, j) to the node (t + 1, k), k E Z(j) (still using the network formulation) has either been eliminated as being non-optimal, or is included in a solution that has been explicitly evaluated, then it is ‘fathomed’. Associated with this concept we define the set K(t, j) as {klk E Z(j) and the arc from (t, j) to (t + 1, k) is unfathomed} for each t, j. We assume again that J={l,2 ,...) n}. Outline of branch-and-bound algorithm (1) Set t = 0, p = 1; let j0 E .Zbe the unique index such that Xf&, 0) = 1; set K(0, jO)= I&). (2) Solve the substitute problem (discussed below) for search over periods t + 1, t + 2,. . . , T, with searcher at jt prior to the search, and with searcher location in the tirst period (i.e. t + 1) restricted to K(t, j,). Obtain thereby (also discussed later) a lower bound p on the optimal probability of no detection when searcher paths are restricted to those passing through jO, JI,...,ZP (3) If P
T.3. STJYMKF
134
jo,il,
* - - jr). The probability of no detection on optimal continuation of the path is given by the product of 9
Prlno detection up to t on path jotjt , . . . , jt] and Pr[no detection on the optimal continuation jt+r,. . . , jTI no detection on path jo,jr,. . . , jJ. The first term can be computed from jo, jI , . . . , j t (and is unity for f = 0). Thus the lower bound on probability is obtained from a lower bound on the second term only, the associated search problem beii that of the substitute problem. For this reason we need discuss the substitute problems and their ~s~iated botmds for the case of t = 0 only. The other eases follow directly from the preceeding remarks. (Clearly the bench-and-Lund ~go~t~ must then also provide for updating of relevant probabiities.) The first approach is suggested directly by the network form of constraint. The true objective may be viewed as minimizationof expected probability of no detection conditional on w, which is a re-interpretation of (6), written as: Q = x@p,[pr{uo det~~onl~get
path = o}].
Let us now rather consider rn~i~tiou of the expectation of the logarithm of this conditional probab~ity. The objective thus becomes mi~mi~tion of:
where
is the probability that the target path passes through cell k at time t. This is now a linear objective function, and together with the network constraints forms a standard linear network flow problem which can be solved directly. A bound may be obtained from Jensen’s inequality. To see this, let us express equation (6) in the following simpiifi~ form:
(where no,[y(*,a,+)Iis the condition probab~ity of detection when the target path is w and searcher motion is defined by y(*,0,a)).Then let y*(*,e,-) represent optimal searcher motion and let y’(1, ) represent searcher motion when the above substitute problem is being solved. We thus have log Q* L x,pw
log ~~[y*(*,1,a)](by Jensen’s in~qu~i~) (9)
where the second inequality follows by optima&y in the substitute problem. ~~o~unately it appears that the above ineq~i~ is too weak to be usefully effective in the branch-and-Lund procedure. We have included discussion of this approach here partly to warn
Search for a movingtargetwhen searchermotion is restricted
I35
off potential users, but also because of the interesting property noted in the next section, viz that the first full solution generated is usefully near-optimal. This suggests a potential heuristic procedure for large problems (i.e. following the branch-and-bound procedure only until t = T for the first time). The practicality of such a procedure depends heavily, however, on the efficiency of the network algorithm used, and although we have not as yet been able to implement the suggestion in any computationally attractive way, it was considered sufficiently interesting to warrant mention. The second approach is to use as substitute problem in the branch-and-bound procedure a search problem with relaxed constraints, i.e. in step (2) of the branch-and-Lund ~gori~rn we solve a search problem with moving target, with constraints on resource availability per period, but not on searcher motion as such. Essentially, this probelm requires choice of one cell each period (since we require X(i, t) E (0, 1}), where choice in period T (t + 1 d r G T) is limited to that reachable from j,, i.e. choice must be made from a set 1, where:
&+I= KO, iA J, = ulj E I(k) for some k E .LJ. It is not required here, however, that j7 should be a member of Ii’jl-l). If the resource indivisi~ity constraint is also relaxed (i.e. if we allow 0 C X(i, t) 6 1 rather than X(‘j, t) E (0,1)) then a solution can be obtained nu~ri~aIly by using Brown’s[ If algorithm. Experience has, however, shown that this further relaxation results in bounds which are also too weak for practical use. Brown’s algorithm can still be used while the indivisibility constraint is retained, but owing to the non-convexity of the feasible region defined by the indivisibility constraint, the solution obtained satisfies only necessary (but not sufficient) conditions for optimality. Thus optimality in the substitute problem cannot be guaranteed, and the implied bound on the original problem may be incorrect (which may result in the elimination of an optimal continuation). As in other non-convex problems, the chances of identifying the true optimal solution can be improved by repeating Brown’s algorithm for a number of different initial plans. Our experience is, however, that this approach yields little or no practical improvement. In fact, by using Brown’s agony for the substitute problem with the ‘myopic’ initial plan as suggested in [I], we have achieved optimality or very near-optim~ity with the branch-ad-Lund approach in ah of a large number of test cases. In over 100 test cases in which the true optimal solution was known, this solution was found in all except 4 of the cases, and in the remaining cases the probability of no detection never exceeded the minimum probability by more than 0.41% of its value. For these reasons we propose use of Brown’s algorithm for the substitute problem with retention of the indivisibility constraint, although it must be conceded that this gives a good heuristic rather than strictly optimal solution. Retention of indivisibility in fact simplifies Brown’s algorithm substantially. For the sake of completeness, the steps of this simplified dlgorithm are outlined below in the nomenclature of the present paper. (Recall that in the ~gorithm the target path proba~iIities are conditions on failure of the search over the searcher path jet, jr ,. . . ,j*; again for Markov motion these path probabilities are easily generated recursively.) Brown’s scheme for the relaxed problem (1) Start with an arbitrary set of search locations i(T) E 1, for f + I S T 5 T. Select a suitable small positive number E. Set B = 1 and T = t + 1, and go to step (2). (2) Evaluate for each k E Jf:
where S(i, j) is the usual Kronecker delta. CAOR
Vol. 6 No. 3-S
T. J. STEWART
136
(3) Replace i(7) by j satisfying
(I- exP (-&)HYj) = Maxkl{l - exp (-#3&P(k)]. If r c ‘I’,set T = 7 f 1 and return to step (2). Otherwise proceed to step (4). (4) Compute 6 = kz P(k) +PMT)l =P
[-i%(T)&
kM *F f
necessity 6 S @.If P - 6 < E, then stop: within au accuracy of E, f$ is the lower bound on probab~ity of no detection on connation; f~he~ore we may set jr+,= i(f + 1) in the blah-Ed-Lund procedure. If P - 0 I 6, then set @= 0, T = f + 1 and return to step (2). As shown in the next section, this approach is very suitable for the branch-and-bound procedure; in some examples this approach has reduced by a factor of about ld the total near of branches to be evaluated, and by a factor of about 108the number of solutions explicitly enumerated. This yields a practical solution of the single-searcher(indivisibleresource) case. In the case of multiple searchers, the number of branches required in the corresponding branch-and-bound procedure increases ex~nenti~y with N, and this approach would soon cease to be viable. When N is too small for the infinitely divisible algorithm to be used satisfactorily, a convenient heuristic approach is to allocate searchers sequentially, using the single-searcherbranch-andbound procedure for the allocation of each in turn, but with p~bab~ties taken conditional on no detection by the previously allocated searchers. We cannot make any rigorous statement on closeness to optimahty of this heuristic. Nevertheless, apart from its intuitive appeal, such a ‘greedy’ approach (in which the greatest marginal value is obtained at each successive notation) has some jus~~tion here for N small in ~omp~son with the number of cells to be searched, since in such cases there is little overlap of effort between searchers.
Of
S.ACOMPUTATIONALEXAMPLE
As has been indicated, we restrict attention here to those cases where there is limited divisibility of resources. A vehicle particularly well-suited for comparing the various approaches is a variation of the problem suggested by McCabef4],viz. the problem of search for a one~~ensio~ random walker. The example is of some interest in itself in that it is a one-diiensional analogue of a typical counter-ins~gency search problem in which the search patrol would first proceed to the scene of some event and thereafter spread out, in pursuit of the insurgents, whose movements are so unpredictable a priori as to be effectively a random walk from the point of view of the searcher. The solution to the on~~ension~ case can suggest the rate at which search should spread out from the origin, even in more general cases. Thus we consider a target whose position at time f, t L -r for some given r > 0, is given by xr, where: xwr=O Pr(xt+,--x*= l)=Pr(X~+,-x~ =-I)=;. In order to keep the model simple we shall assume that r is an even number. This will imply that crossings of the target path and a searcher path only occur if and when both searcher and searched arrive at the same location at the same time. Only at such crossings is detection possible. At this point we depart from McCabe’smodel in two respects. Firstly we allow N searcher paths, each of which satisfies the following for i = 1,. . . ,N: 441
E(-191)
]#$$*I- Cpi,lS 1 for f2 1
Search for a moving target when searcher motion is restricted
137
where & is the position of searcher i at time t. (Search thus starts at t = 1, with all searchers
orighally at the origin.) Secondly, instead of certain detection when paths cross (requiring & = 00for all j in our model), we assume a certain overlook probability a, i.e. & = -In a for all j. We compare the behaviours of the various approaches as they are applied to this problem, seeking to maximize probability of detection within time T. For computational efficiency and for ease of applying the results of the previous sections, it is preferable at each time t to work in terms of t + r + 1 ‘cells’, where cell j corresponds to position [-(t + r)+ 20’- l)] on the random walk. The target path probabilities pU can be derived from: (1) probability that the target is in cell j at t = 1, which is r+l
(I’:)(:) .
-
;
(2) Markov transition probabilities: Pr[target is in cell j at time t + 11target is in cell i at time t] =iforj=i,i+l = 0 otherwise. The searcher constraints of Section 2 are defined by: (1) X((r/2)+l,O)=N =0 Xti, 0) (2) Z(j) = b, j + 1).
j#(r/2)+1;
We consider firstly use of the method of Section 3, modified as suggested at the beginning of Section 4 (i.e. the network form of the convex-simplex method, used in spite of the limited divisibility of resources). Not surprisingly, little success was had for N = 1, but this situation is in any case of little interest here, as the branch-and-bound procedure yields a useful solution. For purposes of comparison the cases N = 2 and N = 4 were considered for time horizons T = 10 and T = 20. Results with overlook probability a = 0.2 are shown in Table 1 for three different starting values for searcher paths. For N/2 of the searchers the following were the three starting values (the others being mirror images of these): starting path (a): 0, 1, 0, -1, 0,. . . starting path (b): 0, 1, 2, 3,4,. . . starting path (c): 0, 1,0,- 1, 0, 1,2, 1,. . . (with change of direction whenever l&l is a Fibonacci number-a suggestion by McCabe). Table 1. Probabilities of no detection for multiplesearcherswith plansselectedaccordingto daerent schemes
t N-2 Scheme
Modified infinitedivisibility algorithm
Sequential use of single-searcher branch-and-bound algorithm
initial path
N=4
1 = 10
T = 20
T = 10
T = 20
(a)
0.09471
0.06403
0.02051
0.01229
(b)
0.09228
0.07484
0.01556
0.00920
(cl
0.09391
0.08456
o.a2051
0.00545
-
0.06537
0.01878
0.00691
0.00232
T. J.
138
STEWART
The results in Table 1 are compared with those obtained by sequentially allocating searchers by the branch-and-bound procedure, as suggested at the end of Section 4. Two conclusions are evident. Firstly with these numbers of searchers the modified infinite-divisibility algorithm performs considerably less well than the sequential branch-and-bound procedure. Secondly performance of the algorithm is strongly dependent on the starting solution, but no single starting solution is consistently best. If thus appears that for N of these magnitudes this algorithm is of little use. Computer running times were, however, less than the computer time required for the branch-and-bound approach, and appeared to be but little influenced by the value of N. In the branch-ad-bond procedures the total number of branches to be evaluated (i.e. the m~imum number of substitute problems to be solved) is (2 + 4 -t * - * + ZT-‘) = 2(2’-’ - l), while the total number of possible solutions is 2’. The efficiencies of the two potential approaches to the branch-and-bound procedure can be compared in terms of the numbers of each of these problems which have actually to be evaluated. In using the first approach and the bound in (9), it was found that for T = 10 the total number of substitute problems to be solved was 848 (out of a maximum of 1022), and 594 (out of 1024) full solutions had to be enumerated. Even when an approximate bound was used, based on a second-order Taylor expansion for log T_, for which expectations were taken (which adds a variance term to the right-hand side of (9)) this did not improve the situation substantially. Clearly it is about as ~ompu~tion~ly efficient to enumerate and evaluate all possible searcher paths. This first approach is thus of no value as a branch-and-Lund procedure, but a most interesting property is illustrated in Table 2. This table gives for a single searcher, for T = 10 and T = 20, and for overlook probability of 0.2, the probability of no detection on the optimal searcher path and on the first paths selected by each of two branch-and-bound approaches (the second discussed below). It is seen that the first selection with the first approach is considerably better than the first selection with the second. approach. What is more important, the result of the former approach compares well with the solution obtained by the branch-and-bound algorithm. It is this fact which suggests that this first selection may form the basis of a heuristic solution procedure which may be useful in some cases. Practical implementation of the procedure is very strongly dependent on the av~abili~ of a very efficient algorithm for the network flow problem (probably an ~go~t~ in which use is made of special s~ctures}. A standard genes-pose out-of-kiher algorithm proved to be not very efficient, and its use led to computation times for obtaining the tirst selection which were higher than those for obtaining the final solution by the second branch-and-bound approach. This difficulty has not yet been resolved. The second approach to the branch-and-bound procedure, based on using the relaxed problems for bounding and for branch selection, was found to be considerably more efficient. For T = 10 it required 105 (out of 1022) substitute problems to be solved and only 8 (out of 1024) solutions to be fully enumerated. For larger T the reduction was far more dramatic. The numbers required for T = 20 were 430 substitute problems and 13 full solutions (compared with 2M)and for T = 30 these were 1276 and 16 (compared with 2’“). This is a feasible approach to the solution of the problem of a single searcher with cons~ained motion.
Table2. Probabilities of no detection for a sin&e searcher For first path generated procedure
in the branch-and-bound Optimal
T 1st substitute problem (minimizing Eilog probability])
10 20
-
2nd substitute problem (relaxed constraints)
Solution
0.2348
0.2827
0.20740
0.1284
0.1858
0.08514
Search for a moving target when searcher motion is restricted
Fig. 1. Optimal searcher paths for T = 20and T = 30 and for overlook probability = 0.2.
Fig. 2. Optimal searcher paths for T = 20 and for overlook probabilities a = 0.05,O.u) and 0.50.
The compu~tion~ times on the CDC CYSRR 174 system were about 40 set for 2’ = 20 and 140 set for 7’ = 30. (The optimal solutions were in fact found within one-third to oneq~er of these times, the remaining time being required to confirm optimality). Finally in view of the intrinsic interest in this example, some solutions obtained with the branch-and-bound algorithm are displayed in Figs. I and 2. In Fig. 1 are shown the paths for T = 20 and 30 and overlook probability 0.2. It appears that a uniformly optimal solution does not exist, as these paths depend on the time horizon. In Fig. 2 the paths for three different overlook probabilities (a = 0.5, 0.2 and 0.05) are shown for T = 20. As could have been anticipated, the rate of expansion of search effort is slower for the higher overlook probabilities (when no detection is less conclusive). 6. CONCLUSIONS has been shown that the problem of opts search with constrained searcher motion can be solved numeric~y in the discrete time and space situation for infinitely divisible resources. It
140
T. J.
STEWART
When resources are indivisible, the heuristic branch-and-bound procedure provides a means of computing good plans in the form of a searcher track, although optimality cannot be guaranteed or proved in general. The intermediate case has not been investigated in depth, but a suggestion has been made as to how such problems may be handled as sequences of indivisible resource problems.
REFERENCES 1. Scott S. Brown, Optimal search for a moving target in discrete space and time. Tech. Rep., Daniel H. Wagner, Associates, Paoli, Pennsylvania (1977).(Submitted for publication.) 2. Anthony P. Ciervo, Search for moving targets. Pacific-Sierra Research Corp., Santa Monica, California, Rep. 619A (1976). 3. Markku Lukka, On the optimal searching tracks for a moving target. SIAMJ. Appl. Math. 32, 126-132(1977). 4. Bernard J. McCabe, Searching for a one-dimensional random walker. J. Appl. Pd. 11,&G93 (1974). 5. Michel Simmonard, Linear Programming.Prentice-Hall, Engkwood Cliffs, N.J. (1966). 6. Lawrence D. Stone, Theory of Optimal !&wch. Academic Press. New York & London (1975). 7. Willard 1. Zangwill, Nonlinear Progmmming: A UnijiedApproach. Prentice-Hall, Engiewood Cliffs, N.J. (PM).