- Email: [email protected]
Optimization of testability parameters
Computers ind. Engng Vol. 17, Nos 1-4, pp. 333-339, 1989 Printed in Great Britain. All rights reserved
OPTIMIZATION
OF T E S T A B I L I T Y
Nael Computer
0360-8352/89 $3.00 + 0.00 Copyright © 1989 Pergamon Press plc
PARAMETERS
A. Aly
I n f o r m a t i o n Systems and P r o d u c t i o n M a n a g e m e n t C a l i f o r n i a State University, S t a n i s l a u s Turlock, C a l i f o r n i a 95380
Department:
ABSTRACT The main o b j e c t i v e of this study was to investigate p e r f o r m a n c e models of a u t o m a t i c d i a g n o s t i c systems taking into c o n s i d e r a t i o n its i m p e r f e c t i o n s such as incorrect i s o l a t i o n and false alarms. This was a c c o m p l i s h e d by d e v e l o p i n g an o p t i m i z a t i o n model to assist the decision maker in d e t e r m i n i n g the optimal values of t e s t a b i l i t y p a r a m e t e r s which maximize his/her utility function. This will provide %:he d e c i s i o n maker with a tool to e v a l u a t e the p a r a m e t e r s set forth by the d e s i g n e r and to assess the real c a p a b i l i t y of t h e d i a g n o s t i c system. This tool also will help the d e c i s i o n maker to check if the correct d e t e c t i o n and i s o l a t i o n c a p a b i l i t y of the system, as well as the imperfections of the system, e.g., incorrect isolation and false alarms are a c c e p t a b l e and satisfactory. An i n t e r a c t i v e p r o g r a m was d e v e l o p e d to help implement the o p t i m i z a t i o n model. This p r o g r a m p r o v i d e s the d e c i s i o n maker with enough f l e x i b i l i t y to evaluate d i f f e r e n t s t r a t e g i e s and to repeat the decision process after c h a n g i n g one or more of the model's parameters. An example will be p r e s e n t e d to show the a p p l i c a t i o n s of this o p t i m i z a t i o n model. INTRODUCTION In recent years the d e v e l o p m e n t and use of automatic systems as m a i n t e n a n c e tools for a v i o n i c s and a i r b o r n e e q u i p m e n t / s y s t e m s has increased appreciably. Automatic fault-detection/fault-isolation (FD/FI) systems which use B u i l t - I n Test: (BIT) can be an important aid to system m a i n t a i n a b i l i t y and system availability by e l i m i n a t i n g the need for t i m e - c o n s u m i n g manual t r o u b l e s h o o t i n g techniques. The m a i n t e n a n c e system for avionics and airborne e l e c t r o n i c equipment works as follows: At the operational site, e l e c t r o n i c e q u i p m e n t is tested using BIT to detect and isolate any line r e p l a c e a b l e unit (LRU) that fails. The faulty LRU then is removed from the system, and a spare is substituted so the system can resume operation. The faulty LRU is sent to the i n t e r m e d i a t e / s h o p level where the faulty module or shop r e p l a c e a b l e unit is isolated and replaced. The e x p e r i e n c e with a u t o m a t i c d e t e c t i o n and isolation systems, in the form of BIT, has not: lived up to expectations. Failure to detect, failure to isolate, incorrect isolations, false alarms, false isolation, and can not d u p l i c a t e (CND) are reported as a result of system d i a g n o s t i c inadequacies. These pro b l e m s are e x p l a i n e d as follows (Aly and Aly, 1988): Failure to detect. the failure.
An LRU
fails,
but
the BIT
fails
to detect
or report
Failure to isolate. An LRU fails, and the failure is d e t e c t e d by BIT. However, BIT fails to isolate the faulty LRU and reports no failure. Incorrect isolation. However, BIT isolates False
alarms.
False LRU.
isolation.
The BIT
An LRU fails, and the failure is d e t e c t e d a good LRU instead of the faulty one. reports
As a result
a failure, of a false
333
while alarm,
the e q u i p m e n t BIT
isolates
by BIT.
is good. a good
334
Proceedings of the 1lth Annual Conference on Computers & Industrial Engineering Cannot duplicate. A f t e r the B I T m i s t a k e n l y r e p o r t s a f a i l u r e a l a r m ) , B I T r e p o r t s no f a u l t y L R U in the i s o l a t i o n p r o c e s s .
(false
W i t h the e x i s t e n c e of s u c h p r o b l e m s , the e v a l u a t i o n of the o p e r a t i o n a l c a p a b i l i t y of B I T d i a g n o s t i c s y s t e m b e c o m e s a real c h a l l e n g e . E v e n t h o u g h m a n y r e a s o n a b l e m e a s u r e s of e f f e c t i v e n e s s to e v a l u a t e the p e r f o r m a n c e of B I T h a v e b e e n d e v e l o p e d (Aly and B r e d e s o n , 1983; G l e a s o n , 1981; H o r k o v i c h , 1981; and T u t t l e a n d L o v e l e s s , 1980), they do not c o n s i d e r the full i m p a c t of d i a g n o s t i c s y s t e m e r r o r s . C o n s e q u e n t l y , t h e r e is a n e e d for a m e t h o d o l o g y to h e l p the d e c i s i o n m a k e r to a s s e s s the real c a p a b i l i t y of B I T d i a g n o s t i c s y s t e m s that w i l l a l s o c o n s i d e r the e r r o r s t h a t o c c u r f r o m B I T inadequacies. PERFORMANCE
MEASURES
FOR DIAGNOSTIC
SYSTEMS
T w o m e a s u r e s of e f f e c t i v e n e s s w e r e d e v e l o p e d (Aly, 1985) to a s s e s s the real c a p a b i l i t y of BIT. The f i r s t is f a l s e i s o l a t i o n e r r o r , ~ e r r o r , w h i c h r e p r e s e n t s f a l s e i s o l a t i o n w h e n a g o o d L R U is m i s t a k e n l y i s o l a t e d as a r e s u l t of a f a l s e alarm. T h e s e c o n d is the f a i l u r e to d i a g n o s e (detect a n d / o r isolate) error, 8 e r r o r , w h i c h is d e s i g n e d to s h o w the a c t u a l F D / F I c a p a b i l i t y . L e t f a i l u r e r a t e be (1), rate of c o r r e c t d e t e c t i o n and i s o l a t i o n of i n c o r r e c t i s o l a t i o n be ([i) , f a l s e a l a r m rate be (If), c a n n o t r a t e be (Ic) a n d r a t e of r e p a i r be (p), then
li (if-k c )
"
I(~÷I i ÷Zi)
be (li), r a t e duplicate
(1)
÷ l i ( 2 U ÷ If - I c)
The s m a l l e r the v a l u e of e the b e t t e r . The w o r s t v a l u e of ~ o c c u r s w h e n all the f a l s e a l a r m s are m i s t a k e n l y i s o l a t e d , i.e., w h e n i c = 0 (no C N D ) , and w i l l be as l a r g e as p o s s i b l e w h e n i i = i, t h e n the m a x i m u m v a l u e of a, ~c is
xf ~C
=
3p ÷ I + i F
(2)
while, x (u
+
Yi)
l ( ~ ÷ l i + Y i ) + l i ( 2 ~ + I f - lC) The s m a l l e r the v a l u e d e n o t e d 8 c is
of
8 the b e t t e r .
5~
The m i n u m u m
÷ k
possible
+ If
(3)
value
of
8,
(4)
It is n o t i c e d that ~ e r r o r is m o r e s e n s i t i v e to f a l s e a l a r m a n d C N D r a t e than to c o r r e c t d e t e c t i o n and i s o l a t i o n and i n c o r r e c t i s o l a t i o n . Also, it is n o t i c e d t h a t 8 is m o r e s e n s i t i v e to the v a r i a t i o n of r a t e of c o r r e c t d e t e c t i o n and i s o l a t i o n and r a t e of i n c o r r e c t i s o l a t i o n t h a n the v a i a t i o n of f a l s e a l a r m and r a t e of CND. F r o m the increase
a b o v e a n a l y s e s , it is o b s e r v e d c o r r e c t d e t e c t i o n and i s o l a t i o n
OPTIMIZATION Model
that: d e c r e a s i n g while
a and 8 e r r o r s decreasing false alarm.
will
MODEL
Structure
The p u r p o s e of this o p t i m i z a t i o n model is to d e t e r m i n e the o p t i m a l t e s t a b i l i ty p a r a m e t e r s w h i c h m a x i m i z e the p e r f o r m a n c e of the a u t o m a t i c t e s t i n g e q u i p m e n t or B I T s ~ s t e m s t a k i n g i n t o c o n s i d e r a t i o n the i m p e r f e c t i o n s of t h e s e s y s t e m s such as f a l s e a l a r m s , f a l s e i s o l a t i o n s , a n d i n c o r r e c t i s o l a t i o n s . H o w e v e r in m a n y real life s i t u a t i o n s , such as in m i l i t a r y a p p l i c a t i o n s , it is a l m o s t i m p o s s i b l e to a s s i g n m o n e t a r y v a l u e s to ~he c o n s e q u e n c e s of t h e s e
Aly: Optimization of testability parameters
335
imperfections. Therefore, it seems more reasonable to rely on a utlility function structure which is assessed by the decision maker to show his/her preference toward the importance of different imperfections according to importance, and the type of the system, as well as the mission involved. This utility function will utilize the two measures of effectiveness: a error, and 8 error. Let f be a function defined on E n, X a subset of E n, and xeX a vector of testability parameters as a function of the failure rate of the equipment, e.g.,
Xl •
, x 2=
and
~
kf , x 4= -~-, ~c and x 5- ~, then , x~ = -~-
x = (li~,
~i/l,
~f/l,
~c/l,
x " (x I ,x 2, .... x 5)
~/l)
Let
fl(x) - a
and f2{x) = S From equations
(i) and
(3)
fl(X)
Xl(Xs'X 4) = Xl÷X2+Xs+Xl(2X5÷X3.X4)
f2(x)
= Xl+X2+Xs+Xl(2Xs+X3.X4)
(5)
and x2+x 5
where
0 5 f i (x) 5 1
for
(6)
i - 1,2
The a£tributes which will be considered here are fl(x), and f2(x) with u(fl(X), fp(x)) as the utility function. The maximization of this utility function m~ans the minimization of the resulting errors from the diagnostic system and its imperfecions. The conditional utility function ul(fl(x)) represents the false alarm and CND, while u2(f2(x)) represents the correct detection and isolation capability of the system in the form of failure to correctly diagnose malfunctions. u(fl(x)) averse.
and u(f2(x))
are monotonically
Therefore,
where
decreasing
alfl(X)
ul(fl(x))
= dl+hl(- e
u2(f2(x))
- d2+h2(-e alf2(x).b2
ai,
bi,
ci
and h i • 0
.bl e
and increasingly
Clfl(x)
)
ec2f2(x))
risk
(7) (8)
, i=I,2
Also, each attribute is utility independent of its complement. Consequently, ul(fl(x)) and u2(f2(x)) are mutually utility independent. Therefore, the two-attribute utility function u(fl(x),f2(x)) is a multilinear in the form: u(fl(x),f2(x))-klul(fl(x))
÷ k2u2(f2(x))
÷ {l-kl-k2)Ul(fl(x))u2(f2{x))
where, ÷
-
k I = U(fl,§2)
(9)
336
Proceedings of the 1lth Annual Conference on Computers & Industrial Engineering
f[ and f~ are the least p r e f e r r e d values of fl(x) and f2(x), respectively, and f[ and f~ are the most p r e f e r r e d values of fl(x) and f2(x), respectively. General
Optimization
Model
The genera] o p t i m i z a t i o n model optimal t e s t a b i l i t y p a r a m e t e r s c o n s t r u c t e d as follows:
Maximize UCfl(x ), Subject
d e s i g n e d to tackle the p r o b l e m of finding the of a u t o m a t i c testing and BIT e q u i p m e n t can be
f2(x))
to XI+X 2 < 1
(I0)
x4-x 3 _< 0
(ii)
fl(x)
_< C 1
(12)
f2(x)
< c2
(13)
Xi > 0 ~
i=I,...,5
where c I is the a c c e p t e d or target c 2 is the a c c e p t e d or targe% value Where,
value of the false i s o l a t i o n error, of failure to d i a g n o s e error.
and
c I = (l-Pl)~ c
and
c2
=
~c + (i"P2)(I"~c)
ac and 6c are the critical values of false isolation error and failure to d i a g n o s e error, respectively. T h e r e f o r e 0~fl(x)&~ c and 8c~f2(x)£1, while Pl and P2 are safety factors e (0-1). The optimal s o l u t i o n of this model produces a vector X that m a x i m i z e s the utility f u n c t i o n p r e s e n t e d in e q u a t i o n (9). Constraint: (i0) is to g u a r a n t e e that the failure rate is greater than or equal to the sum of the rate of correct d e t e c t i o n and isolation, and rate of incorrect isolation. Thus, at any time interval, the number of correct and incorrect isolations after a failure should not exceed the number of failures. C o n s t r a i n t (ii) assures that the rate of false alarm cannot be less than the rate of CND since CND occurs only after false alarm events. C o n s t r a i n t s (12) and (13) allow the d e c i s i o n maker to achieve a targeted value of a and 6 errors w i t h i n a safety margin of their critical values. This o p t i m i z a t i o n model :is a n o n c o n v e x nonlinear p r o g r a m and is highly likely to possess local solutions. The G e n e r a l i z e d Reduced Gradient, GRG2, is used to solve this model. Practical
Implementation
of the O p t i m i z a t i o n
Model
The above o p t i m i z a t i o n model could be i m p l e m e n t e d to help the d e c i s i o n makers of a u t o m a t i c d i a g n o s t i c s and BIT to e v a l u a t e their d i a g n o s t i c systems. The d e c i s i o n maker's main interest: is in the best c o m b i n a t i o n of t e s t a b i l i t y parameters, such a FD/FI, false alarms and incorrect isolations the system will give. The f o l l o w i n g steps are re q u i r e d for an a p p r o p r i a t e i m p l e m e n t a tion of the o p t i m i z a t i o n model. I. The d e c i s i o n maker is asked to state the p r e f e r e n c e toward u n c e r t a i n t y and risk, and q u a n t i f y it ,into two utility functions for both the false isolation, and failure to d i a g n o s e errors. Then, a c o m p o s i t e u t i l i t y f u n c t i o n as ~n e q u a t i o n 9 will be developed. 2. The critical values of the measures of e f f e c t i v e n e s s (false isolation error, a c and failure to d i a g n o s e error, 8c) should be d e t e r m i n e d using e q u a t i o n s 2 and 4.
Aly: Optimization of testability parameters
337
3. The d e c i s i o n maker should specify the safety factors, Pl and P2 for both false i s o l a t i o n and failure to diagnose errors. These factors will show how far from the critical values the d e c i s i o n maker wants to operate. 4. Having e s t a b l i s h e d all the n e c e s s a r y p a r a m e t e r s for the model, d e c i s i o n maker may choose any of the following two options.
the
a) O p t i o n 1 : This option is a p p l i c a b l e if there is a t a r g e t e d value for correct d e t e c i o n and isolation (FD/FI) and a d e c i s i o n maker who is i n t e r e s t e d in finding the optimal testability parameters. These p a r a m e t e r s are p r o p o r t i o n of false alarm a c t i o n s / e v e n t s (which represents the percentage of m a i n t e n a n c e actions that is caused by false alarms) and p r o p o r t i o n of CND a c t i o n s / e v e n t s (which is the p e r c e n t a g e of false alarms that cannot be duplicated) to achieve the specified value of FD/FI. These values will m a x i m i z e the d e c i s i o n maker's utility function, and system performance. This o p t i o n will provide the d e c i s i o n maker with the optimal t e s t a b i l i t y parameters c o m b i n a t i o n when correct d e t e c t i o n and isolation is the main concern. b) Option 2: In this case, there is a certain value of the proportion of false alarm actions that need not be exceeded and the d e c i s i o n maker is interested in finding the rest of the testability parameters. This will result in p a r a m e t e r s g u a r a n t e e d to maximize the decision maker's utility function as well as system p e r f o r m a n c e when false alarm is the main concern. 5. After d e t e r m i n i n g the optimal c o m b i n a t i o n of t e s t a b i l i t y parameters, the d e c i s i o n maker could e v a l u a t e other strategies. Accordingly, a repeat of the d e c i s i o n process after c h a n g i n g one or more of the model p a r a m e t e r s could be done. This can be a c h i e v e d i n t e r a c t i v e l y using a c o m p u t e r interactive program. This interactive program provides the d e c i s i o n maker with enough f l e x i b i l i t y and ease in implementing the o p t i m i z a t i o n model using the above five steps. Numerical
Example
A numerical example is presented to demonstrate the practical of the o p t i m i z a t i o n p r o c e d u r e and the impact of the results.
implementation
Assume the utility function for the false isolation error, ul(fl(x)) failure to d i a g n o s e error, u2(f2(x)) for a risk aversion d e c i s i o n maker are as follows: 1.18Xi+ 32X 1
ul(fl(x)) - 2.359 -.221 (e
u2(f2(x))
5.15e"
)
" 1.11 ~- -.033(e 3"027x2+ 2.39el'436x2]
Let the false alarm weight k I = .40, and the correct diagnosis w e i g h t k 2 = .55. This will put more emphasis on correct diagnosis. Consequently, e q u a t i o n 9 becomes:
U(fl(X),f2(x))
" 1.718-.1012e -.022e
1.18£1Cx)
3.027f2(x)
-.5213e
-.0527e
1.436f2{x)
..O004e1'I8fi(x)+3"o27f2 (x) +.O009el'lSfl(x)+l'426£2(x) ÷.0019e
.32fl(x)+3.027f2(x)
+.o04Se'32£lCx)+l'436f2Cx) CAIE 17-114--W
.32fl(X)
338
Proceedings of the 1lth Annual Conference on Computers & Industrial Engineering
In this example, at any i t e r a t i o n the o p t i m i z a t i o n
e m p h a s i s is on correct diagnosis, and the optimal s o l u t i o n means the best local optimal solution o b t a i n e d after solving model with i0 d i f f e r e n t starting solutions.
Given that the targeted value of correct d e t e c i o n and isolation (FD/FI) is .97, it is r e q u i r e d to find the optimal false alarm (If/l), where If is the rate of false alarm and I is the failure rate, and the optimal p r o p o r t i o n of false alarm a c t i o n s / e v e n t s . From equations 2 and 4, a c = .50, and 8 c = . 125. For each iteration of the d e c i s i o n process, which is p r e s e n t e d in Table i, input the safety factors Pl and P2 such that 0
Iteration
Safety factor for false removal
(Pi)
False
Safety factor for failure to diagnose
(P 2)
Alarm Levels
(FD/FI =
.97)
Optimal false alarm rate per unit failure rate (~f/X)
Optimal % of false alarm actions
% Improvement from upper bound utility function
1
0
.99
3.675
78.61
86.70
2
.35
.94
1.777
63.99
91.72
3
.50
.92
1.255
55.66
93.8
4
.70
.90
0.819
45.04
95.74
S
.82
• 86
0.134
1].04
99.24
6
1.00
.82
0.00
00.00
99.$2
Iteration the false of almost
6 shows that if the d e c i s i o n maker wants to e l i m i n a t e c o m p l e t e l y alarms, P2 should be at least .82 with a failure to d i a g n o s e error .28 w h i c h might not be acceptable.
CONCLUSIONS An o p t i m i z a t i o n model is d e v e l o p e d to find the optimal c o m b i n a t i o n of testability parameters. This model provides the d e c i s i o n maker with a flexible d e c i s i o n process through an i n t e r a c t i ve computer program. S t a r t i n g with certain safety factors for false removal and failure to diagnose, the optimization model will give the optimal c o m b i n c a t i o n of t e s t a b i l i t y p a r a m e t e r s under these conditions. Since increasing one factor can only be done at the expense of d e c r e a s i n g the other, the d e c i s i o n maker has the option of changing either one, consequently, either correct d e t e c t i o n and i s o l a t i o n FD/FI can be improved or false alarm level (if/l) could be reduced. The values of failure to d i a g n o s e and false isolation errors will vary according to the c o m b i n a t i o n of t e s t a b i l i t y parameters. However, using the opt i m i z a t i o n model, the d e c i s i o n maker can always be aware of the system performance measures of e f f e c t i v e n e s s for any optimal c o m b i n a t i o n of t e s t a b i l i t y parameters.
Aly: Optimization of testability parameters
339
REFERENCES
A.A. Aly, J.F. Bredeson, "Analytical procedures for testability", RADC-TR83-4, Final Technical Report, Air Force Systems Command, Rome, New York, USA (1983). N.A. Aly, "Performance models of testability", Ph.D dissertation, University of Oklahoma, Norman, Oklahoma (1985). N.A.AIy, A.A. Aly, "Measures of testability for automatic diagnostic systems", IEEE Transactions on Reliability, Voi.37, No.5, December, pp. 531-538, (1988). D. Gleason, "A measure of BIT/ETE effectiveness", Working Paper, Air Force Systems Command, Rome, New York, USA (1981).
RADC-RBET,
J.A. Horkovich, "Automatic fault detection/fault isolation systems: ments and testing", AFTEC Logistics Assessment Procedures Division, AFC, New Mexico, USA (1981).
RequireKirtland
D.E. Tuttle, R. Loveless,"Built-in-test and external tester reliability characteristics", RADC-TR-80-32, Air Force Systems Command, Rome, New York, USA (1980).
OPTIMIZATION
OF T E S T A B I L I T Y
Nael Computer
0360-8352/89 $3.00 + 0.00 Copyright © 1989 Pergamon Press plc
PARAMETERS
A. Aly
I n f o r m a t i o n Systems and P r o d u c t i o n M a n a g e m e n t C a l i f o r n i a State University, S t a n i s l a u s Turlock, C a l i f o r n i a 95380
Department:
ABSTRACT The main o b j e c t i v e of this study was to investigate p e r f o r m a n c e models of a u t o m a t i c d i a g n o s t i c systems taking into c o n s i d e r a t i o n its i m p e r f e c t i o n s such as incorrect i s o l a t i o n and false alarms. This was a c c o m p l i s h e d by d e v e l o p i n g an o p t i m i z a t i o n model to assist the decision maker in d e t e r m i n i n g the optimal values of t e s t a b i l i t y p a r a m e t e r s which maximize his/her utility function. This will provide %:he d e c i s i o n maker with a tool to e v a l u a t e the p a r a m e t e r s set forth by the d e s i g n e r and to assess the real c a p a b i l i t y of t h e d i a g n o s t i c system. This tool also will help the d e c i s i o n maker to check if the correct d e t e c t i o n and i s o l a t i o n c a p a b i l i t y of the system, as well as the imperfections of the system, e.g., incorrect isolation and false alarms are a c c e p t a b l e and satisfactory. An i n t e r a c t i v e p r o g r a m was d e v e l o p e d to help implement the o p t i m i z a t i o n model. This p r o g r a m p r o v i d e s the d e c i s i o n maker with enough f l e x i b i l i t y to evaluate d i f f e r e n t s t r a t e g i e s and to repeat the decision process after c h a n g i n g one or more of the model's parameters. An example will be p r e s e n t e d to show the a p p l i c a t i o n s of this o p t i m i z a t i o n model. INTRODUCTION In recent years the d e v e l o p m e n t and use of automatic systems as m a i n t e n a n c e tools for a v i o n i c s and a i r b o r n e e q u i p m e n t / s y s t e m s has increased appreciably. Automatic fault-detection/fault-isolation (FD/FI) systems which use B u i l t - I n Test: (BIT) can be an important aid to system m a i n t a i n a b i l i t y and system availability by e l i m i n a t i n g the need for t i m e - c o n s u m i n g manual t r o u b l e s h o o t i n g techniques. The m a i n t e n a n c e system for avionics and airborne e l e c t r o n i c equipment works as follows: At the operational site, e l e c t r o n i c e q u i p m e n t is tested using BIT to detect and isolate any line r e p l a c e a b l e unit (LRU) that fails. The faulty LRU then is removed from the system, and a spare is substituted so the system can resume operation. The faulty LRU is sent to the i n t e r m e d i a t e / s h o p level where the faulty module or shop r e p l a c e a b l e unit is isolated and replaced. The e x p e r i e n c e with a u t o m a t i c d e t e c t i o n and isolation systems, in the form of BIT, has not: lived up to expectations. Failure to detect, failure to isolate, incorrect isolations, false alarms, false isolation, and can not d u p l i c a t e (CND) are reported as a result of system d i a g n o s t i c inadequacies. These pro b l e m s are e x p l a i n e d as follows (Aly and Aly, 1988): Failure to detect. the failure.
An LRU
fails,
but
the BIT
fails
to detect
or report
Failure to isolate. An LRU fails, and the failure is d e t e c t e d by BIT. However, BIT fails to isolate the faulty LRU and reports no failure. Incorrect isolation. However, BIT isolates False
alarms.
False LRU.
isolation.
The BIT
An LRU fails, and the failure is d e t e c t e d a good LRU instead of the faulty one. reports
As a result
a failure, of a false
333
while alarm,
the e q u i p m e n t BIT
isolates
by BIT.
is good. a good
334
Proceedings of the 1lth Annual Conference on Computers & Industrial Engineering Cannot duplicate. A f t e r the B I T m i s t a k e n l y r e p o r t s a f a i l u r e a l a r m ) , B I T r e p o r t s no f a u l t y L R U in the i s o l a t i o n p r o c e s s .
(false
W i t h the e x i s t e n c e of s u c h p r o b l e m s , the e v a l u a t i o n of the o p e r a t i o n a l c a p a b i l i t y of B I T d i a g n o s t i c s y s t e m b e c o m e s a real c h a l l e n g e . E v e n t h o u g h m a n y r e a s o n a b l e m e a s u r e s of e f f e c t i v e n e s s to e v a l u a t e the p e r f o r m a n c e of B I T h a v e b e e n d e v e l o p e d (Aly and B r e d e s o n , 1983; G l e a s o n , 1981; H o r k o v i c h , 1981; and T u t t l e a n d L o v e l e s s , 1980), they do not c o n s i d e r the full i m p a c t of d i a g n o s t i c s y s t e m e r r o r s . C o n s e q u e n t l y , t h e r e is a n e e d for a m e t h o d o l o g y to h e l p the d e c i s i o n m a k e r to a s s e s s the real c a p a b i l i t y of B I T d i a g n o s t i c s y s t e m s that w i l l a l s o c o n s i d e r the e r r o r s t h a t o c c u r f r o m B I T inadequacies. PERFORMANCE
MEASURES
FOR DIAGNOSTIC
SYSTEMS
T w o m e a s u r e s of e f f e c t i v e n e s s w e r e d e v e l o p e d (Aly, 1985) to a s s e s s the real c a p a b i l i t y of BIT. The f i r s t is f a l s e i s o l a t i o n e r r o r , ~ e r r o r , w h i c h r e p r e s e n t s f a l s e i s o l a t i o n w h e n a g o o d L R U is m i s t a k e n l y i s o l a t e d as a r e s u l t of a f a l s e alarm. T h e s e c o n d is the f a i l u r e to d i a g n o s e (detect a n d / o r isolate) error, 8 e r r o r , w h i c h is d e s i g n e d to s h o w the a c t u a l F D / F I c a p a b i l i t y . L e t f a i l u r e r a t e be (1), rate of c o r r e c t d e t e c t i o n and i s o l a t i o n of i n c o r r e c t i s o l a t i o n be ([i) , f a l s e a l a r m rate be (If), c a n n o t r a t e be (Ic) a n d r a t e of r e p a i r be (p), then
li (if-k c )
"
I(~÷I i ÷Zi)
be (li), r a t e duplicate
(1)
÷ l i ( 2 U ÷ If - I c)
The s m a l l e r the v a l u e of e the b e t t e r . The w o r s t v a l u e of ~ o c c u r s w h e n all the f a l s e a l a r m s are m i s t a k e n l y i s o l a t e d , i.e., w h e n i c = 0 (no C N D ) , and w i l l be as l a r g e as p o s s i b l e w h e n i i = i, t h e n the m a x i m u m v a l u e of a, ~c is
xf ~C
=
3p ÷ I + i F
(2)
while, x (u
+
Yi)
l ( ~ ÷ l i + Y i ) + l i ( 2 ~ + I f - lC) The s m a l l e r the v a l u e d e n o t e d 8 c is
of
8 the b e t t e r .
5~
The m i n u m u m
÷ k
possible
+ If
(3)
value
of
8,
(4)
It is n o t i c e d that ~ e r r o r is m o r e s e n s i t i v e to f a l s e a l a r m a n d C N D r a t e than to c o r r e c t d e t e c t i o n and i s o l a t i o n and i n c o r r e c t i s o l a t i o n . Also, it is n o t i c e d t h a t 8 is m o r e s e n s i t i v e to the v a r i a t i o n of r a t e of c o r r e c t d e t e c t i o n and i s o l a t i o n and r a t e of i n c o r r e c t i s o l a t i o n t h a n the v a i a t i o n of f a l s e a l a r m and r a t e of CND. F r o m the increase
a b o v e a n a l y s e s , it is o b s e r v e d c o r r e c t d e t e c t i o n and i s o l a t i o n
OPTIMIZATION Model
that: d e c r e a s i n g while
a and 8 e r r o r s decreasing false alarm.
will
MODEL
Structure
The p u r p o s e of this o p t i m i z a t i o n model is to d e t e r m i n e the o p t i m a l t e s t a b i l i ty p a r a m e t e r s w h i c h m a x i m i z e the p e r f o r m a n c e of the a u t o m a t i c t e s t i n g e q u i p m e n t or B I T s ~ s t e m s t a k i n g i n t o c o n s i d e r a t i o n the i m p e r f e c t i o n s of t h e s e s y s t e m s such as f a l s e a l a r m s , f a l s e i s o l a t i o n s , a n d i n c o r r e c t i s o l a t i o n s . H o w e v e r in m a n y real life s i t u a t i o n s , such as in m i l i t a r y a p p l i c a t i o n s , it is a l m o s t i m p o s s i b l e to a s s i g n m o n e t a r y v a l u e s to ~he c o n s e q u e n c e s of t h e s e
Aly: Optimization of testability parameters
335
imperfections. Therefore, it seems more reasonable to rely on a utlility function structure which is assessed by the decision maker to show his/her preference toward the importance of different imperfections according to importance, and the type of the system, as well as the mission involved. This utility function will utilize the two measures of effectiveness: a error, and 8 error. Let f be a function defined on E n, X a subset of E n, and xeX a vector of testability parameters as a function of the failure rate of the equipment, e.g.,
Xl •
, x 2=
and
~
kf , x 4= -~-, ~c and x 5- ~, then , x~ = -~-
x = (li~,
~i/l,
~f/l,
~c/l,
x " (x I ,x 2, .... x 5)
~/l)
Let
fl(x) - a
and f2{x) = S From equations
(i) and
(3)
fl(X)
Xl(Xs'X 4) = Xl÷X2+Xs+Xl(2X5÷X3.X4)
f2(x)
= Xl+X2+Xs+Xl(2Xs+X3.X4)
(5)
and x2+x 5
where
0 5 f i (x) 5 1
for
(6)
i - 1,2
The a£tributes which will be considered here are fl(x), and f2(x) with u(fl(X), fp(x)) as the utility function. The maximization of this utility function m~ans the minimization of the resulting errors from the diagnostic system and its imperfecions. The conditional utility function ul(fl(x)) represents the false alarm and CND, while u2(f2(x)) represents the correct detection and isolation capability of the system in the form of failure to correctly diagnose malfunctions. u(fl(x)) averse.
and u(f2(x))
are monotonically
Therefore,
where
decreasing
alfl(X)
ul(fl(x))
= dl+hl(- e
u2(f2(x))
- d2+h2(-e alf2(x).b2
ai,
bi,
ci
and h i • 0
.bl e
and increasingly
Clfl(x)
)
ec2f2(x))
risk
(7) (8)
, i=I,2
Also, each attribute is utility independent of its complement. Consequently, ul(fl(x)) and u2(f2(x)) are mutually utility independent. Therefore, the two-attribute utility function u(fl(x),f2(x)) is a multilinear in the form: u(fl(x),f2(x))-klul(fl(x))
÷ k2u2(f2(x))
÷ {l-kl-k2)Ul(fl(x))u2(f2{x))
where, ÷
-
k I = U(fl,§2)
(9)
336
Proceedings of the 1lth Annual Conference on Computers & Industrial Engineering
f[ and f~ are the least p r e f e r r e d values of fl(x) and f2(x), respectively, and f[ and f~ are the most p r e f e r r e d values of fl(x) and f2(x), respectively. General
Optimization
Model
The genera] o p t i m i z a t i o n model optimal t e s t a b i l i t y p a r a m e t e r s c o n s t r u c t e d as follows:
Maximize UCfl(x ), Subject
d e s i g n e d to tackle the p r o b l e m of finding the of a u t o m a t i c testing and BIT e q u i p m e n t can be
f2(x))
to XI+X 2 < 1
(I0)
x4-x 3 _< 0
(ii)
fl(x)
_< C 1
(12)
f2(x)
< c2
(13)
Xi > 0 ~
i=I,...,5
where c I is the a c c e p t e d or target c 2 is the a c c e p t e d or targe% value Where,
value of the false i s o l a t i o n error, of failure to d i a g n o s e error.
and
c I = (l-Pl)~ c
and
c2
=
~c + (i"P2)(I"~c)
ac and 6c are the critical values of false isolation error and failure to d i a g n o s e error, respectively. T h e r e f o r e 0~fl(x)&~ c and 8c~f2(x)£1, while Pl and P2 are safety factors e (0-1). The optimal s o l u t i o n of this model produces a vector X that m a x i m i z e s the utility f u n c t i o n p r e s e n t e d in e q u a t i o n (9). Constraint: (i0) is to g u a r a n t e e that the failure rate is greater than or equal to the sum of the rate of correct d e t e c t i o n and isolation, and rate of incorrect isolation. Thus, at any time interval, the number of correct and incorrect isolations after a failure should not exceed the number of failures. C o n s t r a i n t (ii) assures that the rate of false alarm cannot be less than the rate of CND since CND occurs only after false alarm events. C o n s t r a i n t s (12) and (13) allow the d e c i s i o n maker to achieve a targeted value of a and 6 errors w i t h i n a safety margin of their critical values. This o p t i m i z a t i o n model :is a n o n c o n v e x nonlinear p r o g r a m and is highly likely to possess local solutions. The G e n e r a l i z e d Reduced Gradient, GRG2, is used to solve this model. Practical
Implementation
of the O p t i m i z a t i o n
Model
The above o p t i m i z a t i o n model could be i m p l e m e n t e d to help the d e c i s i o n makers of a u t o m a t i c d i a g n o s t i c s and BIT to e v a l u a t e their d i a g n o s t i c systems. The d e c i s i o n maker's main interest: is in the best c o m b i n a t i o n of t e s t a b i l i t y parameters, such a FD/FI, false alarms and incorrect isolations the system will give. The f o l l o w i n g steps are re q u i r e d for an a p p r o p r i a t e i m p l e m e n t a tion of the o p t i m i z a t i o n model. I. The d e c i s i o n maker is asked to state the p r e f e r e n c e toward u n c e r t a i n t y and risk, and q u a n t i f y it ,into two utility functions for both the false isolation, and failure to d i a g n o s e errors. Then, a c o m p o s i t e u t i l i t y f u n c t i o n as ~n e q u a t i o n 9 will be developed. 2. The critical values of the measures of e f f e c t i v e n e s s (false isolation error, a c and failure to d i a g n o s e error, 8c) should be d e t e r m i n e d using e q u a t i o n s 2 and 4.
Aly: Optimization of testability parameters
337
3. The d e c i s i o n maker should specify the safety factors, Pl and P2 for both false i s o l a t i o n and failure to diagnose errors. These factors will show how far from the critical values the d e c i s i o n maker wants to operate. 4. Having e s t a b l i s h e d all the n e c e s s a r y p a r a m e t e r s for the model, d e c i s i o n maker may choose any of the following two options.
the
a) O p t i o n 1 : This option is a p p l i c a b l e if there is a t a r g e t e d value for correct d e t e c i o n and isolation (FD/FI) and a d e c i s i o n maker who is i n t e r e s t e d in finding the optimal testability parameters. These p a r a m e t e r s are p r o p o r t i o n of false alarm a c t i o n s / e v e n t s (which represents the percentage of m a i n t e n a n c e actions that is caused by false alarms) and p r o p o r t i o n of CND a c t i o n s / e v e n t s (which is the p e r c e n t a g e of false alarms that cannot be duplicated) to achieve the specified value of FD/FI. These values will m a x i m i z e the d e c i s i o n maker's utility function, and system performance. This o p t i o n will provide the d e c i s i o n maker with the optimal t e s t a b i l i t y parameters c o m b i n a t i o n when correct d e t e c t i o n and isolation is the main concern. b) Option 2: In this case, there is a certain value of the proportion of false alarm actions that need not be exceeded and the d e c i s i o n maker is interested in finding the rest of the testability parameters. This will result in p a r a m e t e r s g u a r a n t e e d to maximize the decision maker's utility function as well as system p e r f o r m a n c e when false alarm is the main concern. 5. After d e t e r m i n i n g the optimal c o m b i n a t i o n of t e s t a b i l i t y parameters, the d e c i s i o n maker could e v a l u a t e other strategies. Accordingly, a repeat of the d e c i s i o n process after c h a n g i n g one or more of the model p a r a m e t e r s could be done. This can be a c h i e v e d i n t e r a c t i v e l y using a c o m p u t e r interactive program. This interactive program provides the d e c i s i o n maker with enough f l e x i b i l i t y and ease in implementing the o p t i m i z a t i o n model using the above five steps. Numerical
Example
A numerical example is presented to demonstrate the practical of the o p t i m i z a t i o n p r o c e d u r e and the impact of the results.
implementation
Assume the utility function for the false isolation error, ul(fl(x)) failure to d i a g n o s e error, u2(f2(x)) for a risk aversion d e c i s i o n maker are as follows: 1.18Xi+ 32X 1
ul(fl(x)) - 2.359 -.221 (e
u2(f2(x))
5.15e"
)
" 1.11 ~- -.033(e 3"027x2+ 2.39el'436x2]
Let the false alarm weight k I = .40, and the correct diagnosis w e i g h t k 2 = .55. This will put more emphasis on correct diagnosis. Consequently, e q u a t i o n 9 becomes:
U(fl(X),f2(x))
" 1.718-.1012e -.022e
1.18£1Cx)
3.027f2(x)
-.5213e
-.0527e
1.436f2{x)
..O004e1'I8fi(x)+3"o27f2 (x) +.O009el'lSfl(x)+l'426£2(x) ÷.0019e
.32fl(x)+3.027f2(x)
+.o04Se'32£lCx)+l'436f2Cx) CAIE 17-114--W
.32fl(X)
338
Proceedings of the 1lth Annual Conference on Computers & Industrial Engineering
In this example, at any i t e r a t i o n the o p t i m i z a t i o n
e m p h a s i s is on correct diagnosis, and the optimal s o l u t i o n means the best local optimal solution o b t a i n e d after solving model with i0 d i f f e r e n t starting solutions.
Given that the targeted value of correct d e t e c i o n and isolation (FD/FI) is .97, it is r e q u i r e d to find the optimal false alarm (If/l), where If is the rate of false alarm and I is the failure rate, and the optimal p r o p o r t i o n of false alarm a c t i o n s / e v e n t s . From equations 2 and 4, a c = .50, and 8 c = . 125. For each iteration of the d e c i s i o n process, which is p r e s e n t e d in Table i, input the safety factors Pl and P2 such that 0
Iteration
Safety factor for false removal
(Pi)
False
Safety factor for failure to diagnose
(P 2)
Alarm Levels
(FD/FI =
.97)
Optimal false alarm rate per unit failure rate (~f/X)
Optimal % of false alarm actions
% Improvement from upper bound utility function
1
0
.99
3.675
78.61
86.70
2
.35
.94
1.777
63.99
91.72
3
.50
.92
1.255
55.66
93.8
4
.70
.90
0.819
45.04
95.74
S
.82
• 86
0.134
1].04
99.24
6
1.00
.82
0.00
00.00
99.$2
Iteration the false of almost
6 shows that if the d e c i s i o n maker wants to e l i m i n a t e c o m p l e t e l y alarms, P2 should be at least .82 with a failure to d i a g n o s e error .28 w h i c h might not be acceptable.
CONCLUSIONS An o p t i m i z a t i o n model is d e v e l o p e d to find the optimal c o m b i n a t i o n of testability parameters. This model provides the d e c i s i o n maker with a flexible d e c i s i o n process through an i n t e r a c t i ve computer program. S t a r t i n g with certain safety factors for false removal and failure to diagnose, the optimization model will give the optimal c o m b i n c a t i o n of t e s t a b i l i t y p a r a m e t e r s under these conditions. Since increasing one factor can only be done at the expense of d e c r e a s i n g the other, the d e c i s i o n maker has the option of changing either one, consequently, either correct d e t e c t i o n and i s o l a t i o n FD/FI can be improved or false alarm level (if/l) could be reduced. The values of failure to d i a g n o s e and false isolation errors will vary according to the c o m b i n a t i o n of t e s t a b i l i t y parameters. However, using the opt i m i z a t i o n model, the d e c i s i o n maker can always be aware of the system performance measures of e f f e c t i v e n e s s for any optimal c o m b i n a t i o n of t e s t a b i l i t y parameters.
Aly: Optimization of testability parameters
339
REFERENCES
A.A. Aly, J.F. Bredeson, "Analytical procedures for testability", RADC-TR83-4, Final Technical Report, Air Force Systems Command, Rome, New York, USA (1983). N.A. Aly, "Performance models of testability", Ph.D dissertation, University of Oklahoma, Norman, Oklahoma (1985). N.A.AIy, A.A. Aly, "Measures of testability for automatic diagnostic systems", IEEE Transactions on Reliability, Voi.37, No.5, December, pp. 531-538, (1988). D. Gleason, "A measure of BIT/ETE effectiveness", Working Paper, Air Force Systems Command, Rome, New York, USA (1981).
RADC-RBET,
J.A. Horkovich, "Automatic fault detection/fault isolation systems: ments and testing", AFTEC Logistics Assessment Procedures Division, AFC, New Mexico, USA (1981).
RequireKirtland
D.E. Tuttle, R. Loveless,"Built-in-test and external tester reliability characteristics", RADC-TR-80-32, Air Force Systems Command, Rome, New York, USA (1980).