Subjective assessment of mental workload — A fuzzy linguistic multi-criteria approach

Fuzzy Sets and Systems 62 (1994) 155-165 North-Holland

155

Subjective assessment of mental workload- A fuzzy linguistic multi-criteria approach Tian-Shy Liou Department of Industrial Engineering and Management, National Kaohsiung Institute of Technology Kaohsiung, Taiwan

M a o - J i u n J. W a n g Department of Industrial Engineering, National Tsin Hua University, Hsin Chu, Taiwan Received April 1993 Revised August 1993

Abstract: In this paper, a fuzzy linguistic multi-criteria method for subjective assessment of mental workload is proposed. The criteria ratings and the corresponding importance weights are assessed in linguistic terms, which are described by fuzzy numbers with triangular and trapezoidal membership function. A fuzzy weighted average algorithm, based on the extension principle, is employed to aggregate these fuzzy numbers. The aggregated fuzzy number is then translated back to linguistics and the overall mental workload is expressed in linguistic terms.

Keywords: Subjective assessment; mental workload; linguistic; multi-criteria; fuzzy weighted average.

1. Introduction

Due to the improved technology in the workplace, most of the jobs in the automated production environment required mental effort rather than physical effort [5]. To assure a better person job fit, valid methods of measuring mental workload are certainly needed [13]. From review of literature, the measures of mental workload can be classified into subjective ratings, behavioral timesharing, and psychophysiological indexes [5]. However, there is no real agreement on adequate measure of mental Correspondence to: Prof. M.-J.J. Wang, Department of Industrial Engineering, National Tsin Hua University, Hsin Chu 300, Taiwan.

workload. A common finding is that subjective measures have been extremely popular in operational settings because of the high face validity and the ease of data collection [18]. Further, subjective measures are relatively inexpensive to obtain, nonintrusive, convenient, and easy to analyze. They are valuable tools in the assessment of mental workload. One of the oldest and most extensively validated subjective workload measure is the Cooper-Harper rating scale [2]. Wierwille and Casali modified the Cooper-Harper rating scale by combining a decision tree with a unidimensional 10-point rating scale [17]. The rating scale goes from (1) very highly desirable (operator mental effort is minimal and the desired performance is easily attainable), through (5) moderately objectionable difficulty (high operator mental effort is required to attain adequate system performance), to (10) impossible (instructed task cannot be accomplished reliably). Sanders [14] and Moray et al. [9] also indicated that workload is not a scalar quantity but a vector quantity associated with multiple dimensions, and Sheridan and Simpson [16] proposed subjective mental workload measure with three dimensions, which are time load, mental effort load, and psychological stress. Further, Reid et al. [11] used these three dimensions to develop a Subjective Workload Assessment Technique (SWAT). Using the SWAT, subjects can rate the workload on each of the three dimensions (each with a 3-point scale), and the ratings are then converted to a single interval scale of workload. Most of the subjective measures of mental workload are using scale method which make the subjects express their feelings through rating scales or questionnaires. However, very often the feelings cannot be represented adequately by

0165-0114/94/$07.00 © 1994--Elsevier Science B.V. All rights reserved SSDI 0165-0114(93)E0234-J

156

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

a precise scale. In other words, it is more desirable to measure subjective mental workload using 'linguistic' values than using numerical scale values. Moreover, Moray et al. [10] indicated that the measurement of subjective workload using linguistic quantifiers can yield precise predictions when carried out using rigorous fuzzy measurement methods. Based on the above arguments, it is necessary to develop a fuzzy multi-criteria evaluation method to integrate various linguistic assessments and weights to assess mental workload. In this paper, a method called 'fuzzy linguistic multi-criteria measure' (FLMCM) is proposed. In the FLMCM method, the criteria ratings of mental workload and their corresponding importance weights are assessed by linguistic terms which can be described by fuzzy numbers. An improved fuzzy weighted average (IFWA) algorithm proposed by Liou and Wang is applied to integrate these fuzzy numbers [7]. Subsequently, a method of translating the fuzzy number back into linguistics is also proposed to obtain the linguistic representation of the overall subjective mental workload.

2. Linguistic variable and fuzzy number For the purpose of application, the concepts of linguistic variable and fuzzy number are introduced. Further, some fuzzy operations with tr-cuts are also presented.

2.1. Linguistic variable Linguistic variable is a variable whose values are words or sentences in natural or artificial language. For example, 'weight' is a linguistic variable if its values are linguistic rather than numerical. The value can be 'very low', 'low', 'medium', 'high', 'very high', etc. These lingusitic values can be represented by the approximate reasoning of fuzzy set theory due to that they are too complex or too ill-defined to be reasonably described in quantitative expressions. In this study, the criteria and their importance weights are considered as linguistic variables, and the linguistic values are utilized to assess the criteria ratings of mental workload and their corresponding importance weights.

2. 2. Fuzzy number and its integral value The fuzzy set theory was introduced by Zadeh [19] to deal with problems in which imprecisely defined criteria were involved. A fuzzy number is a special fuzzy subset of real number. The concept of fuzzy number and its integral value is defined in the following [3, 8]. Definition 2.1. A real fuzzy number A is a fuzzy subset of the real line ~ with membership function fa which possesses the following properties: (a) fA is a continuous function from ~ to a closed interval [0, 1], (b) fa(X) = 0 , for all x e (-o% a]; (c) fz is strictly increasing on [a, b]; (d) fm(X) = 1, for all x e [b, c]; (e) fA is strictly decreasing on [c, d]; (f) fa(X) = 0, for all x e [d, ~), where a, b, c, and d are real numbers. Unless elsewhere specified, it is assumed that A is convex, normal and bounded (i.e. - o 0 < a , d <0o). For convenience, the fuzzy number in Definition 2.1 can be denoted by [a, b, c, d], and the membership function fA of the fuzzy number A = [a, b, c, d] can be expressed as

fA(), J1, fA(X) = fn(x),n I, 0,

a~x<~b, b<~x<~c, c<~x<~d,

(2.1)

otherwise,

where fL :[a, b ] ~ [0, 1], and fR:[c, d ] ~ [0, 1]. From Definition 2.1, it is obvious that fL(x), which is the left membership function of the fuzzy number A, is continuous and strictly increasing on [a, b], and f~(x), which is the right membership function of the fuzzy number A, is continuous and strictly decreasing on [c, d]. The inverse function of fL and fR are denoted by g~ and gR respectively. In the special case, if a = b, then fL(x) = f L ( b ) = 1, and gL(y) = b, for y e [0, 1]. Similarly, if c = d, then fR(x) = fR(c) = 1, and gR(y) = C, for y e [0, 1]. Definition 2.2. A is a fuzzy number with the left membership function fL and the right membership function f aa. Suppose that gL is the inverse function of fL, and gR is the inverse function of

Tian-Shy Liou, Mao-Jiun J. Wang / Subjectiveassessment of mental workload fR. Then the left integral value of A is defined as IL(A) =

157

I(x)

gL(y) dy,

and the right integral value of A is defined as IR(A) = ( ' gR(y) dy. J0 Definition 2.3. If A is a fuzzy number with membership function fA, defined as in (2.1), then the total integral value with the index of optimism k is defined as

I~.(A ) = klR(A ) + (1 - k )IL(A ).

(2.2)

where 1R(A) and IL(A) are the right and left integral values of A respectively, and k c [0, 1]. The index of optimism (k) is representing the degree of optimism of a decision maker. A larger k indicates a higher degree of optimism. More specifically, when k = 0, the total integral value /~(A) which represents a pessimistic decision maker's viewpoint is equal to the left integral value of A, i.e. IL(A). Conversely, for an optimistic decision maker, i.e. k = 1, the total integral value I~:(A) is equal to 1R(A). For a moderate decision maker, with k = 0.5, the total integral value becomes /~5(A) = ½[IR(A) + ?L(A)I. Given a > - ~ and d < o % when b 4=c and fA has two straight line segments in [a, b] and [c, d], then A is a trapezoidal fuzzy number, and denoted by (a, b, c, d; 1). A triangular fuzzy number is a special case of trapezoidal fuzzy number with b = c, in this case, it is denoted by (a, b, d; 1). In this study, linguistic values are represented by triangular and trapezoidal fuzzy numbers due to that it is easy to use and interpret.

0 ~ a 1

x

a 2

Fig. 1. The o:-cut of fuzzy number A : [a~', a~].

called an a-cut of A. Specially, if o: = 0, then

As = {XIfA(X)> OI} U[a, b], where a and b are the infimum and supremum of A , = {XlfA(X)> a~} respectively. Since the membership function of fuzzy number is continuous in R, the a-cut of fuzzy number A is a closed interval, which can be denoted by A,, = [a~, a~]. See Figure 1. Further, the addition and multiplication of fuzzy numbers are defined as follows. Definition 2.5. The addition of fuzzy numbers A and B is a fuzzy number, denoted by A E) B with membership function fA~B(Z) = V

(fz(X) AfB(y)),

z--x+y

where V and A represent the maximum and minimum operators. Proposition 2.1. The oc-cut of A ~ B is [a~+

b~, a~ + bg].

Definition 2.6. Suppose A and B are two nonnegative fuzzy numbers. Then the multiplication of A and B is a fuzzy number, denoted by A ® B, with membership function fA®B, and fZ®B(Z) = V z--x

(fA(X) A f , ( y ) ) . .y

2. 3. Fuzzy operation with tr-cut For convenience, some basic properties of fuzzy number needed in this study are summarized, and more discussions can be found in [6, 20]. Definition 2.4. Suppose that A is a fuzzy number with membership function fA. Then for every c~e[0, 1], the set A , = {XlfA(X)>~ 0l}, is

Proposition 2.2. The [a~. b~, a~. bg].

ol-cut

of

A ®B

is

Definition 2.7. The multiplication of a fuzzy number by an ordinary number k ~ + is a fuzzy number, denoted by kA. The membership function is fkA, where f~A(Z)=fA(x), for z = kx.

Proposition 2.3. The tx-cut of kA is [ka'(, kay.

158

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

To measure the subjective mental workload

Select criteria

I

Assign criteria ratings and veights using linguistic terms

I

These are linguistic values of linguistic variable

Represent the linguistic ratings and weights as fuzzy numbers

I Aggregate these fuzzy numbers: find the fuzzy weighted average

An improved efficient algnrithu is applied

I Obtain an aggregated fuzzy number which represents the overall mental workload

I

Translate the a~regated fuzzy number back to linguistic terms

The integral values and an optimisl index used to rank fuzzy numbers is applied

1 Overall lental workload in linguistic term is obtained Fig. 2. The frameworkof the fuzzylinguisticmulti-criteriamental workloadmeasurementmethod.

3. The fuzzy linguistic multi-criteria measure method

The framework of the fuzzy linguistic multicriteria measure method is shown in Figure 2. The main procedure is illustrated in the following. 3.1. Linguistic representations and assessments

Suppose there is a set of decision criteria C1, C2 . . . . . C, which are utilized to represent different aspects of the mental workload and to each criterion C~, an importance weight W,- is assigned. The criteria ratings are linguistic variables with linguistic values VL, L, M, H,

and VH, where V L = V e r y Low, L = L o w , M = Medium, H = High, and VH = Very High. These linguistic values are treated as fuzzy number with triangular membership function. These membership functions are shown in the following, and the graphic presentations is shown in Figure 3. VL: (0, O, 3; 1) fc(x)=l-~x,

0~
L: (0, 3, 5; 1) ~'lx,

fc(x)=t½(5-x),

0~
Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

fw

159

fw ~

"VeryLow"

1

,._

~w"

1

1

0

3

1 w

0

f

W

w

3

2

lO w

5

-•clium" ,,.._

v

5

8

10 w

"Very High"

1

v

0

5

7

10

W

0

7

10 w

Fig. 3. Membership functions of linguistic values for criteria ratings.

B. U&SL: (0,0,2,4; 1)

M: (2, 5, 8; 1) ~ ( x - 2),

f¢(x) =

(-~(8 - x),

5<~x~<8.

~½(x - 5), = trio-

x),

5~
~lx, fw(x) = 1½(4 - x ) ,

7~
Similarly, the importance weights are linguistic variables with linguistic values U, B.U&SL, SL, B.SL&M, M, B.M&SE, SE, B.SE&VSE, and VSE, where U = Unimportant, B.U&SL=Between Unimportant and Slightly important, SL = Slightly important, B.SL&M = Between Slightly important and Moderately important, M = Moderately important, B.M&SE = Between Moderately important and Seriously important, SE = Seriously important, B.SE&VSE = Between Seriously important and Very Seriously important, and V S E = V e r y Seriously important. These linguistic values are treated as fuzzy numbers with triangular or trapezoidal membership function. These membership functions are shown as follows, and the graphic presentations are shown in Figure 4.

U: (0, 0, 2; 1) f,,(x) = ~ ( 2 - x ) ,

0~
B.SL&M: (0, 2, 5, 7; 1)

VH: (7, 10, 10; 1)

fc(x)=l(x-7),

4-x),

SL: (0, 2, 4; 1)

H: (5, 7, 10; 1)

fc(x)

fw(x)=

&(x) =

½x, 1,

0<~x~<2, 2~
1(7- x),

5~
M: (3, 5, 7; 1) ~½(x- 3), &(x)--t½(7-x),

5~x~7.

B.M&SE: (3, 5, 8, 10; 1) l(x-3), f fw(x) - - / 1 ,

L~(10- x),

3~
5 ~
SE: (6, 8, 10; 1) ~½(x- 6),

6~
fw(x)=[½(lO-x), 8~
O~
3~
fw(x) = t l ,

6~
Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

160

"Unimportant"

f

R

"Between Unimportant & Slightly Important" f R

R

Important"

"Slightly

i

v

0

2

10

R

0

2 4

10

0

R

2

4

lO

R

"Between Slightly Important & "Moderately Important" "Between Moderately f ~loderately Important" f Important & Seriously fR R R Important" 1 1 t i I i i

i ! i ! i i

5

8

I

!

,, o

f

R

2

5

7

10

0

R

5

7

10

"Between Seriously Important & Very Seriously Important"

f

"Seriously Important"

3

R

0

f

R

3

10 R

"Very Seriously Important"

1

6

8

10

R

0

6

8

10 R

8

10

R

Fig. 4. Membership functions of the linguistic value for importance weights. VSE: (8, 10, 10; 1) fw(x)=l(x-8),

8~
For each criterion C~ and its importance weight W~, a linguistic rating value a~ and weighting ~ is assigned respectively, where aie Sc, and wi c Sw, i = 1, 2 . . . . . n, are fuzzy numbers, Sc = {VL, L, M, H, VH}, and S w = {U, B.U&SL, SL, B.SL&M, M, B . M & S E , SE, B.SE&VSE, VSE} are the sets of fuzzy numbers which represent the linguistic values. The ais and rP~s are then aggregated to get the final rating )7.

3. 2. Aggregate the criteria ratings and importance weights

The fuzzy weighted average (FWA) is used to aggregate the criteria ratings a~, a2 . . . . . a, and the importance weights ~q, ~'2. . . . . a,,.

The F W A considers that the weighted average 37 which is the aggregated rating of different criterion ai and importance weight ffi, is calculated by 37 ~ - - f ( a l ,

-

a 2 .....

an,

i~1,

14,'2 . . . .

#1 @ ~'2@...@a,,

, i,~n)

(3.1)

To simplify the procedure of finding the membership function u, of 37, the improved fuzzy weighted average (IFWA) algorithm suggested by Liou and Wang [7] is applied. The I F W A algorithm is summarized in the following: (1) Discretize the range of membership [0, 1] into a finite number of values oq, a'2 . . . . , tr,,. The degree of accuracy depends on m, and 0= tk'l " ~ C I f 2 < " " " < ~ m - - I < ( Y m = 1. (2) For each a~j, find the corresponding interval for ai in xi and rPi in wi, i = 1, 2 . . . . . n. The end-points of these intervals are [ai, b,-] and

161

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

[Ci, di], i = 1, 2 , . . . , n. [ai, bi] and [ci, di] a r e the support of the at-cuts of ~ and ff~ respectively. (3) Find the minimum and the maximum of f(x,,

x~ . . . . .

x n , w , , w~ . . . . .

wn)

W l X 1 "{- W 2 X 2 q" . . . q - W n X n )

wj + w2 + " " . + w~

where xi=a~ or b~, i = 1, 2 . . . . . n. Define fl(Wl )W2 .....

and

w~=c~

or

d~,

J, T = { j I b j < U , j • J } .

(c.3) Take j away from J, where j belongs to T. If J 4: 0, then repeat step (c), else stop step (c) and U is the maximum o f f , . (It has also been proven that U is the maximum of f ( x l . . . . . Xn, Wl . . . . . W,), and the right end-point of the cvj-cut of )7 is obtained.) (d) [L, U] is the interval of ag-cut of )7. The [L, U] is denoted by [lt, ut]. (4) Repeat steps (b) and (c) for every act, and the approximate membership function uy of )7 is obtained.

Wn)

= f ( a l , az . . . . .

an, wl, wz . . . . .

w,)

wla~ + w2a2 + • • • + Wnan )

w~ + wz + . . . + w,, L(w,,

w~ . . . . .

=f(bl,

w,,)

b2 . . . . .

bn, Wl, w2 . . . . .

Wn)

w l b l + w2b2 + • • • + w~bn )

wl+w2+'' ~(w,, w z .....

wn

=J~(w, . . . . .

"+Wn

I d,)

Wk-,, dk, Wk+l . . . . .

Wn),

and fu(W,, w2 . . . . .

w, l dk)

=f~(w, . . . . .

Wk-,, dk, Wk+, . . . . .

W,).

Subsequently, the following computational steps are executed. (a) Evaluate L = f l ( c l ,

The aggregated rating of the mental workload obtained by the I F W A algorithm is a fuzzy number with membership function uy, the approximate membership function uy is shown in Figure 5. It will be more meaningful if the fuzzy weighted average of the mental workload can be represented in linguistic terms.

C 2 , . . . , Cn) and U =

f , ( c , , c2 . . . . . Cn). Let l = { i l a ~ < L , i = l ,

{jlbj>U,j=l,

X n , wl . . . . . wn), and the left end-point of the aj-cut of )7 is obtained [7].) (c) If J = 0 then U is the maximum of f, and stop step (c) else wi =di for i • R, and w~ = ci for i ~ Ru. (c.1) Evaluate U t = f , ( w l , w2 . . . . . wn Idj) for j • J . (c.2) Let U =Um = maxj~j Uj, put m into R, and take m away from

o f f ( x 1 .....

2.....

2.....

n},

J=

n} a n d R ~ = R , =

0. (b) If I = 0 then L is the minimum of fl and stop step (b) else w~=di for i • ~ and wi=c~

3. 3. Translate b a c k to linguistics

To translate membership function back to linguistics is a rather sophisticated problem.

f~

1.2 1.0

(b.1) Evaluate L~=f~(w~, wz . . . . . w, Idi) for/e/. (b.2) Let L L m mini~t L~, put m into R] and take m away from I, T = =

=

{i l a i > L, i • I } .

(b.3) Take i away from I, where i belongs to T. If 1 4: ~, then repeat step (b), else stop step (b) and L is the minimum of fl. (It has been proven that L is the minimum

0.8

0.6

0.4 0.2" 0.0 2

4

6

8

10

Fig. 5. The approximation membership function of y.

162

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

Given the conditions that the interested fuzzy number is convex and normal. Several methods have been proposed (e.g. Schmucker [15], and Eshragh and Mamdami [4]). In this study, an optimism index method which ranks fuzzy numbers with integral value proposed by Liou and Wang [8] is applied.

The translation rules are: (1) the linguistic mental workload is dj, if 3/o = lIT()7) -- IT(j)], (2) the linguistic mental workload is dj+l, if Mo = liT()7) -- ]T(j + 1)1, (3) the linguistic mental workload is between dj and dj÷l, if

3. 3.1. The index of optimism and the integral value Let d/, i = 1, 2 . . . . . n, be the fuzzy numbers with membership functions fa,(x), x belongs to R, i = 1 , 2 , . . . , n . The definition of the left integral value IL(a~) and the right integral value 1R(di) are given bY Definition 2.2. An optimist would rank the a,. by IR(a;), the larger the 1R(a~), the higher the rank. On the contrary, a pessimist would rank the a~ by IL(a~). In g_eneral, it is reasonable to combine IR(a~) and IL(d~) with an index of optimism k, where 0 ~< k <~ 1. A larger k value represents a higher degree of optimism. The total integral value of a~ is IT(ai) = klR(ai) + (1 -- k)IL(a3

where 0 ~< k <~ 1.

This is a convex combination of IR(di) and 1L(di). The concept of total integral value is then utilized to translate the aggregated measurement of mental workload )7 back to linguistic terms.

3. 3. 2 The method of translation The criterion rating 6i is the elements of the linguistic value set Sc = {VL, L, M, H, VH}. Let D = {Cl . . . . , cn} be the set of ratings which is assigned to rating criteria. Then D is a subset of Sc. Let Xmi, = infS, and Xmax= s u p S , where S=~_Jxi¢oSi and Si={xlfx,(x)>O}. Let So = {dl, d2 . . . . . dp}, where di • Sc and {xlf4(x ) > 0} c [Xmi,, Xmax], i = 1, 2 . . . . . p. Then the total integral value IT(d/) of di, i = 1, 2 . . . . . p, and IT()7 ) of )7 can be found. Without losing generality, assume that IT(dl) < lt(d2) < " " < 1T(t~p), ~li • So, i= 1, 2 . . . . , p. Denote IT(i)= IT(d/). Then there exists ] such that/T(]) ~
/140= lIT()7) - ½lIT(j) + IT(j + 1)]l. For instance, if d j = M (Medium), dj+~ = H (High), then the mental workload is 'Medium', if MO=IIT()7)--IT(J)I; the mental workload is 'High', if Mo = lIT()7) --IT(j + 1)l; and the mental workload is 'between Medium and High', if Mo = lIT()7) - ~ [ I T ( j ) + IT(j ÷ 1)11.

3. 4. Calculation of the total integral value IT(d~) and ITOY) 3.4.1. Calculation of lx(di) Since di, d i e S o , is a fuzzy number with triangular membership function denoted by (ai, b~, ci; 1), i = 1, 2 . . . . . p. From Definitions 2.2 and 2.3, the total integral value of d,- is

IT(d) = ½[kc, + b, + (1 -

k)a,],

i = 1, 2 . . . . .

p. (3.2)

3.4.2. Calculationof &OY) From Section 3.2, the I F W A algorithm can find the approximate membership function uy of )7. The points (l~,tri) and (u~,a~i), i = 1, 2 . . . . . m, constitute the approximate left membership function fL and right membership function fR respectively. Hence, the points (tel, li) and (tri, ui), i = 1, 2 . . . . . m, compose the approximate inverse function of fL and fR respectively. By Definition 2.2, the approximation of the integral value of )7 can be calculated using the commonly used Simpson rule [1]. The method of numerical integral is stated in the following: Divide the interval [0, 1] into n sub-intervals, where n is an even number, the length of each sub-interval h is equal to 1/n (i.e. h = 1/n, and m = n + 1). Then, the left integral value IL()7) and right interval value IR()7 ) are IL()7) ~ 13h(l~ + 412 + 213 + 414 + 21s +''"

+ 4/, + ln+l),

(3.3)

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

and /R(37) ~

163

Table 1. The rating scale descriptions of the criteria

~h(u~ + 4U2

+ 2U3 +

4U4+

2U5

+ . • • + 4u~ + u~+~),

(3.4)

Thus, the total integral value of 37 is IT(37) = klR(37) + (1 -- k)lL(37).

(3.5)

4. Example A hypothetical example is designed to demonstrate the computational process of this fuzzy linguistic multi-criteria measure method for subjective assessment of mental workload. The FLMCM procedure and its numerical computations are summarized in the following. Step 1. Select criteria. Suppose nine rating criteria which were used by researchers in the Human Performance Group at NASA-Ames Research Center are considered, which include: (1) task difficulty (Ct), (2) time pressure (C2), (3) performance (C3), (4) mental/sensory effort (C4), (5) frustration level (C5), (6) stress level (C6), (7) fatigue (C7), (8) activity type (C8), (9) physical effort (C9). The descriptions of the nine rating criteria are listed in Table 1. Step 2. Assign criteria ratings and weights. The importance weights are given by the experts and the ratings can be assessed by the subjects who are performing the tasks. This can be done via a simple rating scale (Table 2). The ratings can be assigned according to the descriptions presented in Table 1. In this example, the assumed ratings and weights are given in T a b l e 3. Step 3. Find the l-r(di),d~So. In this

example, S o = { L , M , H , VH}, take k = l . From (3.2), we have (1) dt = L, (at, bt, ct) = (0, 3, 5; 1), and IT(dl) = 2.75,

(2) dz = M,

(a2, b2, c2) = (2, 5, 8; 1),

and

(a3, b3, c3) = (5, 7, 10; 1),

and

l-r(d2) = 5,

(3) d3 = H,

l-r(d3) = 7.25,

Title

Endpoints

ovcrall workload

low, high

Description

the total workload associated with the task, considering all sources and components task difficulty low, high whether the task was easy or demanding, simple or complex, exacting or forgiving time pressure none, rushed the amount of pressure you felt due to the rate at which the task elements occurred. Was the task slow and leisurely or rapid and frantic? performance failure, perfect how successful you think you were in doing what you were asked and how satisfied you were with what you accomplished mental none, the amount of mental and/or percep/sensory impossible tual activity that was required (for effort example, thinking, deciding, calculating, remembering, looking, searching) frustration fulfilled, how insecure, discouraged, irritated, level exasperated and annoyed versus secure, gratified, content, and complacent you felt stress level relaxed, tense how anxious, worried, uptight and harassed or calm tranquil, placid, and relaxed you felt fatigue exhausted how tired, weary, worn out, and alert exhausted or fresh, vigorous, and energetic you felt activity type skill based, the degree to which task required rule based, mindless reaction to well-learned knowledge routines, required the application of based known rules, or required problemsolving and decision-making physical effort none, the amount of physical activity that was impossible required (pushing, pulling, turning, controling, activating, etc.)

(4) d4 = VH, (a4, b4,

C4) =

(7, 10, 10; 1), and

/T(d4) : 9.25.

Step 4. Determine Iv(y). In

here,

suppose

that m = l l , and trt=0, a~:=0.1, a~3= 0.2 . . . . . oqo=0.9, and aql = 1. By the IFWA algorithm the left end-point and right end-point of orj-cuts of 37 are: It = 1.243,

ul = 8.146;

12= 1.589,

u 2 = 7.812;

/3-- 1.933,

u3 = 7.485;

14 = 2.275,

u4 = 7.162;

15 = 2.615,

u5 = 6.845;

2.952,

u6 = 6.533;

16 =

164

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

Table 2. Rating scale being used for assessing mental workload of contributing of each criterion. Remark: VL: Very Low L: Low M: Medium H: High VH: Very High

,2, Time ressure

Table 3. The ratings of criteria and importance weights Rating criteria Ci 1

Rating of criterion t3i M Importance weight ff,~ SE

2

3

4

5

6

M

VH

M

M

L L L H

SE

B.SL&M VSE B.M&SE M SL M SL

17 = 3.288,

u7 = 6.226;

18 = 3.632,

u8 = 5.924;

/9 = 3.956,

u9 = 5.625;

ll0 = 4.287,

uln = 5.331;

ltl = 4.617,

uil = 5.046.

7

8

9

Since k = ~, by (3.3), (3.4), and (3.5), the total integral value of 37 is I-r(37)=4.75. This algorithm can be implemented by a small computer program. Step 5. Translate the membership function back to linguistics. Since Ir(fi0 < IT(37) < IT(d2), the minimum of 11T(37)--I-r(dx)l, 11T(37)-/v(a2)l, and Mo = 1/T(37) - l[1T(at) +/a-(az)]] is Mo = liar(Y) - iv(a2)l = 0.25. Since a2 = M, therefore, one can conclude that the overall mental workload is 'Medium'.

5. Extension of the FLMCM

Suppose there are n assessors (D~, D2 . . . . , Dn) to measure the mental workload of a specific subject. Let Rtr be the rating for criterion Ct, given by assessor Dr, and let W~r be the importance weight for criterion C,, given by assessor D r, where t = 1, 2 . . . . . k; j = 1, 2 . . . . , n. Then, for each assessor Dr, Rtr and Wtr, t = 1, 2 . . . . . k, are first aggregated to obtain the total mental workload 37i, J = 1, 2 . . . . . n. These n ~s are further aggregated to a overall mental workload 37. Finally, the overall mental workload 37 is translated back to linguistic term, and the overall mental workload is obtained. Applying the F L M C M method, the computation of tr-cut of 37 is slightly modified as follows. For % e [0, 1], let [air, bq] and [cq, dq] be the trp-cut of Rq and Wq respectively, i = 1, 2 . . . . . k. By the I F W A algorithm, the %-cut of )~ is [lpr, Upr], p = 1, 2 . . . . . n, and 37r is the final total workload given by D# Moreover, since the overall mental workload 37 = ( l / n ) ® 071~372@"" (])37n), hence, by Definitions 2.5 and 2.7, and Proposition 2.3, the %-cut of 37 is lip, up], where lp = (1/n)F,7= ~lpj, and up = (1/n)F,7= t upr, p = 1, 2 . . . . . m. Using these lps and UpS, by (3.3) and (3.4), the total integral value of 37 is obtained, and the linguistic overall mental workload is concluded.

6. Discussion and conclusion

In subjective mental workload measurement, it is more appropriate that the assessment of criteria ratings and importance weights are given in linguistic terms. Also, it is necessary to aggregate these linguistic assessments to obtain an overall determination of workload. In this paper, a method is proposed to obtain the fuzzy weighted average of the linguistic ratings and weights, and then translate the aggregated membership back to linguistics. Sometimes, the importance weights are different from job to job, the weights can be obtained by asking the experts to study the job demands and to determine the importance of different criteria to the workload measurement. The importance weights can be given by either

Tian-Shy Liou, Mao-Jiun J. Wang / Subjective assessment of mental workload

direct assignment or through pairwise comparisons [12]. In here, the weights is assigned in linguistics. However, if the importance weights are assessed in numerical scales (i.e. ~/s are real numbers), and the criteria ratings are in linguistics, the proposed algorithm is still valid. In this situation, the fuzzy weighted average is a fuzzy-arithmetic weighted average, and the left end-points and the right end-points of o~j-cuts of )7 are the arithmetic weighted averages of the left end-points and right end-points of the corresponding c~j-cuts of 6/s respectively, where 6/ is the fuzzy number represents the i-th linguistic criterion. In general, the mental workload is preferably assessed by subjective evaluations. Since the FLMCM proposed in this paper is specifically useful for situations where the subjective workload is assessed in linguistic terms, by the face validity the FLMCM is considered to be superior to other numerical methods. For the membership function of the linguistic values, in addition to the triangular membership functions, some other membership functions, such as the trapezoidal membership function, and the R - L type membership function are applicable. However, for the simplicity and effectiveness in implementation, the triangular type of membership function is applied. Although the method discussed in this paper is primarily designed for subjective assessment of mental workload, it provides a general framework for evaluating multi-criteria decision making problem which involved linguistic assessment of criteria ratings and importance weights. References [1] K.E. Atkinson, An Introduction to Numerical Analysis (John Wiley, New York, 1978). ]2] G. Cooper and R. Harper, The use of pilot ratings in evaluation of aircraft handling qualities, NASA Ames Tech. Rept., NASA TN-D-5153 (1969) Moffet Field, CA., NASA Ames Research Center. [3] D. Dubois and H. Prade, Operations on fuzzy numbers, International Journal of System Science 9 (1979) 617-626. [4] F. Eshragh and E.H. Mandani, A general approach to linguistic approximation, International Journal of Man-Machine Studies 11 (1979) 501-519.

165

[5] B.H. Kantowitz, Mental work, in: D.C. Alexander and B.M. Pulat (eds.), Industrial Erognomics - A Practitioner's Guide (Industrial and Management Press, Norcross, 1985, GA.) pp. 31-38. [6] A. Kaufmann and M.M. Gupta, Introduction to Fuzzy Arithmetic Theory and Applications (Van Nostrand Reinhold, New York, 1985). [7] T.S. Liou and M.J. Wang, Fuzzy weighted average: an improved algorithm, Fuzzy Sets and Systems 49 (1992) 307-315. [8] T.S. Liou and M.J. Wang, Ranking fuzzy number with integral value, Fuzzy Sets and Systems 50 (1992) 247-255. [9] N. Moray. G. Johannesen, R. Pew, J. Rasmussen, A. Sanders and C. Wickens, Final report of the experimental psychology group, in: N. Moray (ed.), Mental Workload: Its Theory and Measurements (Plenum Press, New York, 1979) pp. 101-114. [10] N. Moray, B. Turksen, P. Aidie, D. Drascic, P. Eisen, E. Kruschelnicky, L. Money and H. Schonert, Progress in mental workload measurement, Proceedings of the Human Factors Society 30th Annual Meeting (1986). [11] G. Reid, C. Shingledecker and T. Eggemeier, Applications of conjoint measurement to workload scale development, in: R. Sugarman (ed.), Proceedings of the Human Factors Society 25th Annual Meeting (1981) Santa Monica, CA., Human Factor Society. [12] T.L. Saaty, A scaling method for priorities in hierarchical structures, J. Math. P~ychology 15 (1977) 234-281. [13] M.S. Sanders and E.J. McCormick, Human Factors in Engineering and Design (McGraw-Hill Book Company, New York, 1987). [14] A.F. Sanders, Some remarks on mental load, in: N. Moray (ed.), Mental Workload: Its Theory and Measurement (Plenum Press, New York, 1979) pp. 41-78. [15] K.J. Schmucker, Fuzzy Sets, Natural Language Computations, and Risk Analysis (Computer Science Press, USA, 1985) pp. 19-34. [16] T. Sheridan and R. Simpson, Toward the definition and measurement of the mental workload of transport pilots, FTL Rept. R79-4, Cambridge, MA., Massachusetts Institute of Technology, Flight Transportation Laboratory (1979). [17] W. Wierwill and J. Casali, A validated rating scale for global mental workload measurement applications, in: A. Pope and L. Haugh (eds.), Proceedings of the Human Factors Society 27th Annual Meeting (1983) Santa Monica, CA: Human Factors Society, pp. 129-133. [18] Y. Yeh and C. Wickens, Dissociation of performance and subjective measures of workload, Human Factors 3tl (1988) 111-120. [19] L.A. Zadeh, Fuzzy sets, Information and Control 8 (1965) 338-353. [20] H.J. Zimmermann, Fuzzy Set Theory and its Applications, 2nd ed. (Kluwer, Dordrecht, 1991).