- Email: [email protected]
The integration of motivational information: A conjoint measurement analysis
Acta Psychologica 46 (1980) 2 5 7 - 2 6 9 Q North-ttolland Publishing C o m p a n y
THE INTEGRATION OF MOTIVATIONAL INFORMATION: A CONJOINT MEASUREMENT ANALYSIS *
Geert De SOETE ** State University of Ghent, Ghent, Belgium Accepted June 1980
Within Anderson's (1974a, b, c, 1978b) information integration theory, tlie integration of motivational information was investigated by means of conjoint m e a s u r e m e n t techniques. Eighteen university students were asked to judge hypothetical co-students characterized by three features (intelligence, motivation and the extent to which they study) according to their chances to pass. Both rank order data and ratings were obtained. The orderings of most subjects could be represented very well by an additive model. A polynomial regression procedure was applied to determine the shape of the response function for the ratings. As this function was quite linear for all subjects, the ratings could be said to form an interval scale.
In all kinds of circumstances, judgments are being made about people and situations based on a wide variety of information. Both the clinician, the counselor and the personnel manager have to sum up the client, respectively the job applicant, by integrating several diagnostic data. But also in the daily interaction, people continuousIy attribute to each other qualities, intentions and abilities inferred from certain perceived features. It was originally in connection with the latter area, viz., person perception, that Anderson developed his information integration theory. Soon, his theory was applied successfully to the most diverse areas (Anderson 1974a, b, c, 1978b). In this paper we shall investigate the integration of motivational information by use of conjoint measurement techniques. * I wish to thank Andr6 Vandierendonck for his encouragement and n u m e r o u s helpful suggestions in preparing the manuscript. Ivan Mervielde is also gratefully acknowledged for his critical remarks on an earlier draft. ** Aspirant of the Belgian 'Nationaal Fonds voor Wetenschappelijk Onderzoek'. Presently at the L.L. T h u r s t o n e Psychometric Laboratory, University of North Carolina. Davie Hall 013 A, Chapel Hill NC 27514, U.S.A. 257
258
G. De Soete / Information integration
Anderson's information integration theory With his integration theory Anderson (1974a, b , c , 1978b) wants to describe how several kinds of information are combined to make up a unitary judgment. Two operations are sequentially distinguished: valuation and integration. Valuation deals with the internal dimensional representation of each stimulus component (cf Matthai~s et al. 1976). Indeed, each task requires a preliminary ewduation of the meaning and relevance of each piece of stinmlus information (Anderson 1974b). Suppose the stimulus set consists of the cartesian product of three factors A, B and C, which represent three different facets of the stimuli. Valuation of a stimulus (ai, bj, ck) means that scale valtles.f'~ (ai),f2(bj) and f3(ck) (with fl, f2 and f3 real-valued functions) are assigned to the stimulus components. Once the different pieces of information are evaluated, they can be integrated in an overall judgment: the scale values fl(ai), f2(bj) and .[3(ck) are combined by the integration ftmction I in a latent response:
riik
=
l(f,(ai), f2(bi), fa(ck))
( 1)
which results in a manifest response Rij k by means of the response function M:
Ri/k = M(ri/k)
(2)
A scheme of these processes is presented in fig. 1 which is a modified version of Anderson's (1970, 1974a, 1977) functional measurement diagram. STIMULUS COMPONENTS
ai
PSYCHOLOGICAL C OSTIMULUS MPONENTS
.
~, f l i a i .) f 2 ( b j )
ck
:> f 3 ( c k )
VALUATION FUNCTIONS
MANIFEST
RESPONSE
)
bj
I
LATENT
RESPONSE
N
~- rij k
T
E
G
R
A
INTEGRATION FUN CT ION
T
O
:)' Rij k
R
RESPONSE FUNCT ION
Fig. 1. A scheme of Andersons's information integration theory (this is a modified version of Anderson's (1970, 1974a, 1977) functional m e a s u r e m e n t diagram).
G. De Soete / hfformation integration
259
Integration theorists generally take the valuation for granted and concentrate primarily on the integration. Mervielde (1977) explains this by indicating that valuation itself can be seen as a result of an integration operation. Anderson has repeatedly shown how simple algebraic models are apt to describe this process. Especially additive models were successful. Anderson draws a sharp psychological distinction between what he calls a simple adding and an initial impression weighted averaging model. In the common case of constant weights per factor, both models can respectively be written as: rii k = w l f a ( a i ) + w2f2(b/) + w 3 ~ 3 ( c k )
(3)
and
rijt = 00Io + ViA(ai) + v2f2(b,) + v3f3(cD
(4)
with v0 + vl + v2 + v3 = 1. In eq. (4) I0 stands for the initial impression of the subject. Contrary to the w in (3), the v in (4) are only relative weights. However, it can be proved (cf Sch6nemann et al. 1973) that whenever the data fit the following model: rijk = ~l(ai) + ~b2(bi) + 4~3(ck)
(S)
they also fit eqs. (3) and (4). Consequently, in this paper we do not distinguish between (3) and (4) and we consider the additive model (5) only. Although the additive model has played a dominant role, several kinds of interactive models have, rather sporadically, been found. Integration models are usually validated by means of functional measurement, a methodology developed by Anderson (1970, 1974a, 1977, 1978a) parallel to his integration theory. In most information integration experiments the overt responses consist of ratings, which are analyzed by means of ANOVA. Absence of significant interaction effects is interpreted as evidence for an additive model and for the validity of the assumption that the response scale is an interval scale (cf. Anderson 1977, 1978a; Klitzner and Anderson 1977). If however an interaction is detected, one is never sure whether this is due to the judgmental process itself, or to the nonlinearity of the response function. In this case, it is safer to assume only a monotonic relationship between the latent and manifest responses and to diagnose the model by relying
260
G. De Soete /hLtbrmation integration
on the ordinal characteristics o f the data. This can be done by axiomatic conjoint measurement that provides a num ber of qualitative tests (axioms) which are necessary and ahnost sufficient conditions for the different combination rules (Krantz e t al. t 971; Krantz and Tversky 1971). Once the underlying model is determined in this way, the data can be scaled according to it and the relation between the latent and manifest responses can be empirically investigated by means of polynomial regression.
The integration of motivational information When judging the future performance of a person, people generally have not only an impression o f the skills and abilities of the person, but they also know more or less his motivation. How will this motivational information influence the judgment? Intuitively, the idea of motivation as a multiplier seems appealing. This is for instance apparent from Hull's well known formula: Reaction Potential = Drive X H a b i t . The idea occurs also explicitly in Heider's (1958) attribution theory: Performance = Motivation X Ability (1958: 8 3 ) .
{6)
This proposition has been empirically verified by Anderson and Butzin (1974), together with two derived formulas: Motivation = Performance X (Ability) -1
(7)
Ability = Performance × (Motivation)-'
(8)
ANOVAs on the ratings o f two different groups of subjects indicated for (7) as well as for (8) rather an additive than a multiplicative model. Only eq. (6) has been confirmed. However, if one considers the averaged ratings with regard to (6), as plotted in the first panel of figs. 1 and 2 of Anderson and Butzin (1974), as ordinal data, one can represent them very well according to an additive model by lneans o f ADDALS (de
G. De Socte /Information integration
261
Leeuw et al. 1976) [11. The stress [2] value for both groups is less than 0.0001. Similar results have been obtained by Ullrich and Painter (1974). In their experiment students had to judge job applicants according to their ability for filling a managerial position, given their intelligence, experience and achievement motivation. A conjoint measurement analysis of the results revealed an additive model for several subjects. In order to investigate further the role of motivational information we performed the following experiment. University students were asked to judge the chances to pass of a set of hypothetical co-students which were characterized by their intelligence (l), motivation (M) and the extent to which they studied (S). Working with three factors instead of two has subtantial advantages (Klitzner and Anderson 1977) of which the most important is that a better discrimination between additive and interactive models is possible. Given the findings in person perception and attribution (Anderson 1974b, c, 1978b), we can expect in accordance with the results of Ullrich and Painter (1974), an additive model I + M + S. If however Anderson and Butzin's (1974) proposal Future Performance = Past Performance + Motivation × Ability (1974: 609) is valid, a dual-distributive model S + M X I is to be found because the extent to which one studies is a kind of performance, which in turn results from an interaction between ability and motivation. If on the contrary one considers S as a part of the ability to succeed, then a distributive combination rule (I + S) X M must be expected.
The e x p e r i m e n t
Method Su bjec ts E i g h t e e n first year s t u d e n t s , 7 males and 11 females, all enrolled in the F a c u l t y o f Psychological and E d u c a t i o n a l Sciences at the University o f G h e n t , p a r t i c i p a t e d [ 1 ] We thank Norman H. Anderson for making this data available to us. [2] The stress is a goodness of fit measure which is the square root of the normalized sum of squared deviations between tile optimally transformed data and the model. ADDALS uses Kruskal's (1965) stress formula two.
262
(;. De Soete /In¢brmation in tegration
v o l u n t a r i l y in partial f u l f i l m e n t s o f certain class r e q u i r e m e n t s . No o n e was a c q u a i n t e d e i t h e r with c o n j o i n t m e a s u r e m e n t or with A n d e r s o n ' s i n f o r m a t i o n integration theory. Stimuli T h e s t i m u l i c o n s i s t e d o f d e s c r i p t i o n s o f h y p o t h e t i c a l first year s t u d e n t s , w h i c h were c h a r a c t e r i z e d by t h r e e f e a t u r e s : intelligence (3 levels: IQ = 100, 120, 140), m o t i v a t i o n (3 levels: n o t , weakly, s t r o n g l y m o t i v a t e d ) a n d the e x t e n t to w h i c h t h e y s t u d i e d (2 levels: studies little, m u c h ) . By c o m b i n i n g t h e t h r e e variables factorially, 18 stimuli were o b t a i n e d . I n c l u s i o n of an a d d i t i o n a l s t i m u l u s (IQ = 125, weakly m o t i v a t e d , studies little) for c o n v e n i e n c e of design yielded a t o t a l of 19 stimuli. Procedure As we i n t e n d e d to a n a l y z e the j u d g m e n t s n o n m e t r i c a l l y ( a x i o m analyses) as well as m e t r i c a l l y ( p o l y n o m i a l regressions), it was desirable to g a t h e r n o t o n l y ratings b u t also explicit o r d i n a l data, because ties or m i n o r o r d e r inversions in the ratings due t o r a n d o m j u d g m e n t a l f l u c t u a t i o n s can cause m a n y v i o l a t i o n s against t h e c o n j o i n t m e a s u r e m e n t axioms. W h e n b o t h r a n k o r d e r data and ratings m u s t be o b t a i n e d at t h e same t i m e , it is b e t t e r to ask the s u b j e c t s t o o r d e r t h e stimuli first, o t h e r w i s e t h e r e is a big c h a n c e t h a t t h e y will use t h e i r ratings while r a n k o r d e r i n g t h e stimuli. In o r d e r to be able to i n t r o d u c e a n u m b e r of r e p l i c a t i o n s per subject t h e stimuli were a r r a n g e d in a b a l a n c e d i n c o m p l e t e b l o c k design ( C o c h r a n and Cox 1957: plan 11.32), c o m p o s e d of 19 b l o c k s of I0 stimuli. T h e subjects were r e q u e s t e d to r a n k o r d e r stimuli w i t h i n blocks. In this m a n n e r e a c h s t i m u l u s was j u d g e d ten times, while each pair of stimuli was p r e s e n t e d five times. T h e subjects were r u n in group. E a c h subject received a c o m p u t e r g e n e r a t e d b o o k l e t . On the first page t h e y were i n s t r u c t e d t h a t t h e p u r p o s e o f t h e e x p e r i m e n t was to investigate h o w well first year s t u d e n t s in p s y c h o l o g i c a l and e d u c a t i o n a l sciences were already able to p r e d i c t the success of c o - s t u d e n t s at the university. This mild d e c e p t i o n was inspired u p o n Mervielde ( 1 9 7 7 ) . T h e f o l l o w i n g pages cont a i n e d t h e 19 b l o c k s w h i c h h a d to be r a n k o r d e r e d i n d e p e n d e n t l y a c c o r d i n g t o t h e c h a n c e s t h e described s t u d e n t s h a d to pass at t h e university. T h e o r d e r of t h e b l o c k s as well as t h a t of the s~imuli w i t h i n each b l o c k was r a n d o m i z e d p e r subject. Finally t h e 19 stimuli were p r e s e n t e d again a n d the s u b j e c t s h a d to i n d i c a t e o n an eleven p o i n t scale (going f r o m 0 to 10) h o w m a n y c h a n c e s to ten the s t i m u l u s s t u d e n t s h a d to pass. T h e whole task t o o k o n t h e average 65 m i n u t e s t o c o m p l e t e . A f t e r w a r d s the subjects were asked t o fill in a q u e s t i o n n a i r e a b o u t t h e e x p e r i m e n t . R esu Its
Orderings and agreement a m o n g subjects One s u b j e c t did n o t u n d e r s t a n d t h e task p r o p e r l y . His data were discarded as t h e y were u n u s a b l e . F o r e a c h o f the r e m a i n i n g 17 s u b j e c t s the s t o c h a s t i c a l l y d o m i n a n t o r d e r i n g ( C o o m b s a n d H u a n g 1970) over the 19 stimuli was o b t a i n e d . As e a c h pair
263
G. De Soete /Information integration
of stimuli is replicated five times, a s t i m u l u s x is said to d o m i n a t e stochastically y w h e n e v e r x d o m i n a t e s y in m o r e t h a n t w o of t h e five w i t h i n - b l o c k c o m p a r i s o n s of (x, y). K e n d a l l ' s ( 1 9 5 5 ) c o e f f i c i e n t of c o n c o r d a n c e a m o n g t h e subjects is 0.753. W h e n t e s t e d against t h e null h y p o t h e s i s of n o a g r e e m e n t , this value is highly signific a n t (X 2 (18) = 230.5 2, p < 0.001). H o w e v e r , r e j e c t i o n of this h y p o t h e s i s does n o t i m p l y t h a t t h e r e are n o individual differences. Indeed, t h e pairwise Kendall ( 1 9 5 5 ) tau c o r r e l a t i o n s b e t w e e n t h e several orderings range f r o m 0 . 0 2 4 to 0.959. Conseq u e n t l y , t h e data h a d t o be a n a l y z e d individually.
Consistency and transitivity A first way to assess t h e c o n s i s t e n c y a m o n g the j u d g m e n t s of a single subject is to c o m p u t e D u r b i n ' s ( 1 9 5 1 ) test statistic w h i c h u n d e r the null h y p o t h e s i s t h a t each r a n k i n g in each b l o c k is equally likely, is a p p r o x i m a t e l y d i s t r i b u t e d as a chi-square. F o r each subject, t h e X2-value associated w i t h t h e statistic is listed in the first c o l u m n o f t a b l e 1. As t h e critical value at t h e 0.001 level is 4 2 . 3 1 , t h e null h y p o t h e sis c o u l d be r e j e c t e d for all subjects. A n o t h e r way o f l o o k i n g at t h e c o n s i s t e n c y is b y i n s p e c t i n g t h e s u b s e q u e n t choices o n e a c h pair of stimuli. C o o m b s a n d H u a n g ( 1 9 7 0 : 3 2 8 ) n o t i c e d t h a t w i t h five r e p l i c a t i o n s of a 5 0 / 5 0 choice o n each pair, the d i s t r i b u t i o n of the d o m i n a n t s t i m u l u s is a f o l d e d b i n o m i a l over 3, 4 a n d 5 w i t h a m e a n of 3.44 a n d a s t a n d a r d d e v i a t i o n of 0.371. A significant deviation f r o m c h a n c e at the 0.01 level (one-tail test) for t h e average over t h e ( 1 9 ) = 171 pairs is 3.52 or m o r e . T h e average con-
Table 1 Consistency and transistivity. Subject
x 2 associated with Durbin's statistic
Consistency
Number of circular triads
L A R Q G K J F C E P N D B I M H
162.90 161.43 162.44 161.11 157.96 156.75 155.39 159.00 151.65 153.46 152.18 153.58 150.78 151.28 150.59 143.75 136.01
4.988 4.971 4.965 4.936 4.801 4.790 4.766 4.760 4.760 4.754 4.725 4.714 4.714 4.696 4.667 4.602 4.515
0 0 0 0 1 0 2 0 2 1 4 3 4 4 0 5 7
264
G. De Soete /InJormation inte,gration
sistency per s u b j e c t is p r e s e n t e d in t h e s e c o n d c o l u m n of table 1. F o r all subjects t h e h y p o t h e s i s o f r a n d o m n e s s c o u l d be rejected. Besides e s t i m a t i n g the c o n s i s t e n c y of the data, it is also i m p o r t a n t to evaluate t h e t r a n s i t i v i t y , b e c a u s e , if t r a n s i v i t i t y does n o t hold, the overall o r d e r i n g cons t r u c t e d o u t o f the w i t h i n - b l o c k r a n k i n g s is of no m e a n i n g . T h e t r a n s i t i v i t y of t h e s t o c h a s t i c a l l y d o m i n a n t choices per pair can be assessed b y c o u n t i n g t h e n u m b e r o f circular triads ( K e n d a l l 1955). With 19 stimuli, t h e n l a x i m u m n u m b e r is 285, while the e x p e c t e d n u m b e r a s s u m i n g a 5 0 / 5 0 pairwise c h o i c e is 242. T h e n u m b e r o f circular triads for each s u b j e c t is given in c o l u m n t h r e e of table 1. As is a p p a r e n t f r o m table 1, t r a n s i t i v i t y can be said to h o l d q u i t e well for all subjects.
Independence A c c o r d i n g to K r a n t z and T v e r s k y ' s ( 1 9 7 1 ) f l o w c h a r t for t h e diagnosis of three-fact o r p o l y n o m i a l s in the u n s i g n e d case, o n e has to test for i n d e p e n d e n c e in the first place. F a c t o r l is said to be i n d e p e n d e n t of M for a fixed level of S, if t h e r a n k o r d e r o f I is t h e same at all levels of M. This can be assessed b y c o m p u t i n g K e n d a l l ' s c o e f f i c i e n t o f c o n c o r d a n c e b e t w e e n t h e r a n k orders o f I over all levels of M (Wallsten 1976). Moreover, this can be d o n e at e a c h level o f S. T h e s e c o e f f i c i e n t s revealed t h a t 14 o f t h e 17 s u b j e c t s satisfied t h e i n d e p e n d e n c e r e q u i r e m e n t perfectly. F o r one s u b j e c t , M, f a c t o r M was n o t c o m p l e t e l y i n d e p e n d e n t of I. Since the c o e f f i c i e n t of c o n c o r d a n c e , averaged over the levels of S, a m o u n t e d to 0 . 9 3 9 , o n l y a small v i o l a t i o n was involved. T h e two r e m a i n i n g subjects, D a n d H, v i o l a t e d the a x i o m m o r e seriously. T h e i r l o w e s t averaged coefficients of c o n c o r d a n c e were respectively 0 . 0 7 4 ( f o r S i n d e p e n d e n t of M) a n d 0 . 2 0 3 (for 1 i n d e p e n d e n t of M). In cases like these, o n e has to test for sign d e p e n d e n c e . This was n o t d o n e here because t h e factors h a d n o e n o u g h levels to p e r f o r m t h e o t h e r tests r e q u i r e d in t h e signed case. Double cancellation T h e d o u b l e c a n c e l l a t i o n a x i o m could o n l y be t e s t e d for t h e factors I a n d M, as m i n i m a l l y a 3 X 3 m a t r i x is required. In each test of t h e c a n c e l l a t i o n c o n d i t i o n , six cells are involved. In all t h e d i f f e r e n t tests w h i c h are possible in a given 3 X 3 m a t r i x , o n l y six d i f f e r e n t six cell c o m b i n a t i o n s are involved. "The d o u b l e c a n c e l l a t i o n c o u l d be t e s t e d at each level o f S. C o n s e q u e n t l y , 12 d i f f e r e n t six cell c o m b i n a t i o n s could violate t h e c o n d i t i o n . O n l y o n e s u b j e c t , P, failed to satisfy t h e d o u b l e c a n c e l l a t i o n in o n e single six cell c o m b i n a t i o n . All o t h e r subjects, i n c l u d i n g s u b j e c t s D a n d H, satisfied t h e a x i o m perfectly. Join t independence T h e j o i n t i n d e p e n d e n c e was verified n e x t . F a c t o r s I and M are said to be j o i n t l y i n d e p e n d e n t o f S, if t h e r a n k o r d e r over all c o m b i n a t i o n s o f / a n d M is t h e same at all levels o f S. T h e a g r e e m e n t b e t w e e n t h o s e r a n k orders can again be expressed b y K e n d a l l ' s c o e f f i c i e n t of c o n c o r d a n c e . T h e s e c o e f f i c i e n t s were c a l c u l a t e d for the d i f f e r e n t j o i n t i n d e p e n d e n c e tests. A l t h o u g h m o s t subjects s h o w e d s o m e m i n o r violations, t h e results were very similar to t h o s e of the i n d e p e n d e n c e analyses: serious v i o l a t i o n s o c c u r r e d o n l y w i t h s u b j e c t s D a n d H.
265
G. De Soete / Information integration Table 2 Goodness of fit for ADDALS scalings and results of regression analyses. Subject
Kendall tau
Stress
r2
F-ratio *
L A R Q G K J F C E P N D B I M H
1.000 1.000 0.987 1.000 0.951 0.974 0.941 0.922 0.980 0.937 0.907 0.931 0.951 0.964 0.948 0.927 0.864
0.000 0.000 0.000 0.000 0.002 0.000 0.002 0.049 0.000 0.004 0.000 0.054 0.000 0.001 0.017 0.002 0.002
0.956 0.983 0.782 0.961 0.786 0.912 0.742 0.843 0.838 0.842 0.893 0.847 0.847 0.848 0.890 0.854 0.738
347.21 967.39 57.48 349.26 58.85 166.68 46.11 85.77 82.71 85.52 134.19 88.54 88.27 89.33 130.04 93.39 45.15
* All F-ratios have one degree of freedom for the numerator and 16 for the denominator, except for subject A who forgot to rate one stimulus, so that the dffor the denominator for this subject become 15. All F's are highly significant (p < 0.001).
Model characterization F i f t e e n o f the 17 subjects satisfied t h e i n d e p e n d e n c e , double cancellation and j o i n t i n d e p e n d e n c e a x i o m s very well. C o n s e q u e n t l y , t h e i r j u d g m e n t s could best be d e s c r i b e d by t h e additive m o d e l 1 + M + S. A discussion o f t h e resurts for subjects D and H is p o s t p o n e d for a while. Because an additive c o m b i n a t i o n rule seems a p p r o p r i a t e for m o s t subjects, we have scaled each stochastically d o m i n a n t o r d e r i n g a c c o r d i n g to this m o d e l by m e a n s o f t h e A D D A L S a l g o r i t h m (de L e e u w et al. 1976) [3]. S o m e g o o d n e s s o f fit measures for the additive r e p r e s e n t a t i o n s are r e p o r t e d in table 2. The first c o l u m n conrains t h e Kendall tau correlations b e t w e e n the rank o r d e r data and t h e scale values, while the s e c o n d c o l u m n lists the stress values. The squared stress (see f o o t n o t e 2) is up t o a n o r m a l i z a t i o n f a c t o r t h e loss f u n c t i o n m i n i m i z e d b y t h e algorithm. As is evident f r o m table 2, all. r e p r e s e n t a t i o n s are q u i t e satisfactory, even for subjects D and H. The n o r m a l i z e d scale values associated w i t h the levels o f the t h r e e factors are [3] The maximum number of iterations allowed was set to 30, while the minimum stress improvement required for continuation was set to 0.0001.
G. De Soete /hz¢brmation in tegration
266
Table 3 Normalized AI)DALS scale values. Subject
I1
12
I3
S1
L A R Q G K J F C E P N D B I M tt
-0.216 -0.187 --0.760 --0.942 -0.669 -0,409 -0.369 -0.978 -0.369 -0.802 0.707 1.002 1.167 0.375 -0.914 -0.787 0.000
0.006 0.005 -0,029 0,017 0.074 0.000 0.178 0.291 0.038 0.009 -0.001 -0.097 0.045 -0.058 0.036 0.045 0.001
0.222 0.192 0.788 0.959 0.744 0.409 0.192 0.687 0.331 0.793 0.708 1.099 1.122 0.433 0.950 0.832 0.001
-0.999 --0.288 -0.909 -0,744 --0.710 -1.005 0.276 -0.548 1.009 - 0.803 0.707 -0.267 0.000 0.401 0.576 0.812 0.004
$2
0.999 0.288 0,909 0.744 0.710 1.005 0.276 0.548 1.009 0.803 0.707 0.267 0.000 0.401 0.576 0.812 -0.004
M1
M2
M3
-0.672 1.188 0.272 0.215 0,662 -0.510 -1.138 0.679 0.529 -0.290 0.706 0.508 0.409 0.983 0.413 0.256 -1.203
0.006 0.025 0.001 0.017 ---0.074 0,101 0.023 0.028 -0.121 0.250 --0.001 0.098 0.045 - 0.178 0.206 0.239 0.042
0.677 1.162 0.271 0.198 0.736 0.611 1.161 0.651 0.649 0.540 0.707 0.606 0.455 1.161 0.619 0.495 1.245
p r e s e n t e d in table 3. In o r d e r to allow for b o t h w i t h i n - s u b j e c t and b e t w e e n - s u b j e c t c o m p a r i s o n s , n o t only t h e sum o f scale values per f a c t o r but also the t o t a l sum o f s q u a r e d scale values was m a d e c o n s t a n t for each subject. By i n s p e c t i n g t h e range o f t h e scale values per f a c t o r , one can see t h a t a l t h o u g h all subjects e x c e p t D and H, t o o k a c c o u n t o f the three variables, the relative i m p o r t a n c e o f each f a c t o r varied f r o m s u b j e c t t o subject. This finding justifies our e m p h a s i s o n analyzing the data at t h e individual level. F r o m t h e scale values of subjects H and D, it can be i n f e r r e d t h a t t h e s e individuals only d i f f e r e n t i a t e d the stimuli on t h e basis o f respectively one and t w o factors ( m o t i v a t i o n and intelligence). This has given rise t o r a n d o m o r d e r inversions w h i c h caused the m a n y a x i o m violations o f these subjects. As in these cases one can hardly speak o f an imp!icit c o m b i n a t i o n rule, these subjects c a n n o t be said to disc o n f i r m a t h e o r y w h i c h p r e d i c t s an additive m o d e l .
Validation o f the rating scale The scale values o b t a i n e d by A D D A L S can be regarded as e s t i m a t e s o f the latent responses. As o u t l i n e d in the first s e c t i o n , the latent r e s p o n s e s are t r a n s f o r m e d b y the r e s p o n s e f u n c t i o n i n t o m a n i f e s t responses. If this f u n c t i o n is linear, the manifest responses, i.e., t h e ratings, can be said t o f o r m an interval scale. We have applied a stepwise p o l y n o m i a l regression p r o c e d u r e to d e t e r m i n e the shape of this f u n c t i o n . F o r all subjects a linear regression a c c o u n t e d for a s u b s t a n t i a l part o f t h e variance of the scale values, as is a p p a r e n t f r o m the s q u a r e d c o r r e l a t i o n c o e f f i c i e n t s
G. De Soete / Information integration
267
p r e s e n t e d in the t h i r d c o l u m n of table 2. T h e last c o l u m n o f t h a t t a b l e gives t h e Fr a t i o s associated w i t h t h e best linear fit. In n o case a d d i t i o n of h i g h e r o r d e r polyn o m i a l t e r m s , u p to the t h i r d degree, i m p r o v e d the fit significantly. Since t h e linearity o f t h e r e s p o n s e f u n c t i o n has b e e n assessed by this p r o c e d u r e , t h e ratings could b e c o n s i d e r e d as interval data. As a final c h e c k of t h e validity of o u r t w o m a i n c o n c l u s i o n s , viz., t h e a p p r o p r i ateness of t h e additive m o d e l and the linearity of t h e r e s p o n s e scale, a r e p e a t e d m e a s u r e s A N O V A was p e r f o r m e d on t h e ratings [4]. If b o t h o u r assertions were true, significant m a i n effects a n d n o n s i g n i f i c a n t i n t e r a c t i o n effects of t h e s t i m u l u s f a c t o r s were to be o b t a i n e d , unless of course the a s s u m p t i o n s u n d e r l y i n g the m e t h od were t o o seriously violated. T h e A N O V A results were e x a c t l y as e x p e c t e d . T h e m a i n effects of S, 34 a n d I were highly significant (resp. F ( 1 , 1 6 ) = 2 4 . 4 4 , F ( 2 , 1 6 ) = 16.54, F ( 2 , 1 6 ) = 29.91, all p < 0 . 0 0 1 ) , whereas n o n e of t h e F-values associated w i t h t h e i n t e r a c t i o n t e r m s a p p r o a c h e d significance. These results provide a d d i t i o n a l evidence for t h e validity of o u r conclusions!
Discussion and conclusions Conjoint measurement techniques have proved to be quite useful for determining the appropriate model. Although this approach lacks an adequate error theory, this did not cause any particular difficulty in interpreting the results of the axiom analyses. The judgments of all subjects who differentiated the stimuli on the basis of the three factors, could adequately be represented by an additive model. A question put forward by a referee in connection with this result concerns the power of the present 3 × 3 × 2 design to reject the additive model. Indeed, as in most conjoint measurement applications, the design was minimal, since only such a small design allows for testing the consistency and transitivity at the individual level in a meaningful way. This advantage of the 3 × 3 × 2 design is of course worthless when the design lacks enough power to validate the proposed models. Fortunately, previous experimental applications of conjoint measurement w h e r e the very same design was used show it to have the required power (cf. Coombs and Huang 1970; Ullrich and Painter 1974). Our conclusion to an additive model contrasts with the intuitive idea of motivation as a multiplier and suggests that at least Belgian students integrate motivational information in a very similar manner as other [4] The one missing rating (cf. table 2) was estimated by means of linear regression using the latent responses as obtained by ADDALS.
268
G. De Soete / b~formation integratioJt
kinds of information are processed (Anderson 1974b, c). The discrepancy between the present results and those of Anderson and Butzin (1974) may possibly be attributed to cultural differences, as a recent replication of the Anderson and Butzin study in India, conducted by Singh et al. (1979), evidences. Anderson and his colleagues have very often tried to prove the linearity of the response ftmction by heavily relying on ANOVA techniques (e.g., Anderson 1977, 1978a; Klitzner and Anderson 1977). We, on the contrary, have assessed this linearity and consequently the validity of the response scale, by relying only on the ordinal characteristics o f the data for the model diagnosis and without making preliminary distributional assumptions. This can be very useful for fttrther research in the area. Although some methodologically innovative methods have been applied, we have only been able to prove the descriptil,e validity of the additive model. Le. the model is only an as i f model. What a person really does when making a judgment that is like an addition, is not known! Much more research will be needed to answer this question!
References Anderson, N.H., 1970. Functional m e a s u r e m e n t and psychophysical j u d g m e n t . Psychological Review 7 7 , 1 5 3 - 1 7 0 . Anderson, N.H., 1974a. 'Algebraic models in perception'. 111: E.C. Carterette and M.P. Friedman (eds.), Handbook of perception. Vol. II. New York: Academic Press. pp. 2 1 5 298. Anderson, N.H., 1974b. 'Cognitive algebra: integration theory applied to social attribution'. In: L. Berkowitz (ed.), Advances in experimental social psychology. Vol. VII. New York: Academic Press. pp. 1 - 101. Anderson, N.tt., 1974c. 'Information integration theory: a brief survey'. In: D.H. Krantz, R.C. Atkinson, R.D. Luce aiad P. Suppes (eds.), C o n t e m p o r a r y developments in mathematical psychology. Vol. II. San Francisco: Freeman. pp. 236 305. Anderson, N.tt., 1977. Note on functional m e a s u r e m e n t and data analysis. Perception and psychophysics 2 1 , 2 0 1 - 2 1 5 . Anderson, N.H., 1978a. Measurement of motivation and incentive. Behavior Research Methods and I n s t r u m e n t a t i o n 1 0 , 3 6 0 375. Anderson, N.H., 1978b. 'Progress in cognitive algebra'. In: L. Berkowitz (ed.), Cognitive theories in social psychology. New York: Academic Press. pp. 103 - 1 2 6 . Anderson, N.H. and C.A. Butzin, 1974. Performance = Motivation X Ability: an integrationtheoretical analysis. Journal of Personality and Social Psychology 30, 5 9 8 - 6 0 4 . Cochran, W.G. and G.M. Cox, 1957. Experimental designs. Second edition. New York: Wiley. Coombs, C.H. and L.C. Huang, 1970. Polynomial psychophysics of risk. Journal of Mathematical Psychology 7 , 3 1 7 338.
G. De Soete /InCbrmation integration
269
de Leeuw, J., F.W. Young and Y. Takane, 1976. Additive structure in qualitative data: an alternating least squares method with optimal scaling features. Psychometrika 4 i, 471 503. Durbin, J., 1951. Incomplete blocks in ranking experiments. British Journal of Psychology (Statistical section) 4, 85-90. IIcider, F., 1958. The psychology of interpersonal relations. New York: Wiley. Kendall, M.G., 1955. Rank correlation methods. Second edition. London: Griffin. Klitzner, M.D. and N.H. Anderson, 1977. Motivation X expectancy X value: a functional rneasurement approach. Motivation and Emotion 1,347 365. Krantz, D.H. and A. Tversky, 1971. Conjoint-measurement analysis of composition rules in psychology. Psychological Review 78, 151 169. Krantz, D.H., R.D. Luce, P. Suppes and A. Tversky, 1971. Foundations of measurement. Vol. I. New York: Academic Press. Kruskal, J.B., 1965. Analysis of factorial experiments by estimating monotone transformations of the data. Journal of the Royal Statistical Society (Series B, methodological) 27, 351263. Matthatis, W., G. Ernst, U. Kleinbeck and T. Stoffcr, 1976. Funktionales Messen und kognitive Algebra. Ein methodenkritischer Beitrag zur Entscheidung zwischen Modellen der Informationsiutegration. Teil I: Uebcrlegungen zur Diagnose ener Verkntipfungsregcl aus Paarurtcilen. Archiv fiir Psychologie 128,267 291. Mervielde, I., 1977. Persoonsperccptie als informatieverwerking. Unpublished doctoral dissertation, University of Ghent, Belgium. Sch6nemann, P.H., T. Cafferty and J. Rotton, 1973. A note on additive functional measuremont. Psychological Review 80, 85 87. Singh, R., M. Gupta and A.K. Dalai, 1979. Cultural difference in attribution of performance: an integration-thcorctical analysis. Journal of Personality and Social Psychology 37, 13421351. Ullrich, J.R. and J.R. Painter, 1974. A conjoint-measurement analysis of human judgment. Organizational Behavior and Hu man Performance 12, 50-61. Wallsten, T.S., 1976. Using conjoint-measurement models to investigate a theory about probabilistic information processing. Journal of Mathematical Psychology 14, 144 185.
THE INTEGRATION OF MOTIVATIONAL INFORMATION: A CONJOINT MEASUREMENT ANALYSIS *
Geert De SOETE ** State University of Ghent, Ghent, Belgium Accepted June 1980
Within Anderson's (1974a, b, c, 1978b) information integration theory, tlie integration of motivational information was investigated by means of conjoint m e a s u r e m e n t techniques. Eighteen university students were asked to judge hypothetical co-students characterized by three features (intelligence, motivation and the extent to which they study) according to their chances to pass. Both rank order data and ratings were obtained. The orderings of most subjects could be represented very well by an additive model. A polynomial regression procedure was applied to determine the shape of the response function for the ratings. As this function was quite linear for all subjects, the ratings could be said to form an interval scale.
In all kinds of circumstances, judgments are being made about people and situations based on a wide variety of information. Both the clinician, the counselor and the personnel manager have to sum up the client, respectively the job applicant, by integrating several diagnostic data. But also in the daily interaction, people continuousIy attribute to each other qualities, intentions and abilities inferred from certain perceived features. It was originally in connection with the latter area, viz., person perception, that Anderson developed his information integration theory. Soon, his theory was applied successfully to the most diverse areas (Anderson 1974a, b, c, 1978b). In this paper we shall investigate the integration of motivational information by use of conjoint measurement techniques. * I wish to thank Andr6 Vandierendonck for his encouragement and n u m e r o u s helpful suggestions in preparing the manuscript. Ivan Mervielde is also gratefully acknowledged for his critical remarks on an earlier draft. ** Aspirant of the Belgian 'Nationaal Fonds voor Wetenschappelijk Onderzoek'. Presently at the L.L. T h u r s t o n e Psychometric Laboratory, University of North Carolina. Davie Hall 013 A, Chapel Hill NC 27514, U.S.A. 257
258
G. De Soete / Information integration
Anderson's information integration theory With his integration theory Anderson (1974a, b , c , 1978b) wants to describe how several kinds of information are combined to make up a unitary judgment. Two operations are sequentially distinguished: valuation and integration. Valuation deals with the internal dimensional representation of each stimulus component (cf Matthai~s et al. 1976). Indeed, each task requires a preliminary ewduation of the meaning and relevance of each piece of stinmlus information (Anderson 1974b). Suppose the stimulus set consists of the cartesian product of three factors A, B and C, which represent three different facets of the stimuli. Valuation of a stimulus (ai, bj, ck) means that scale valtles.f'~ (ai),f2(bj) and f3(ck) (with fl, f2 and f3 real-valued functions) are assigned to the stimulus components. Once the different pieces of information are evaluated, they can be integrated in an overall judgment: the scale values fl(ai), f2(bj) and .[3(ck) are combined by the integration ftmction I in a latent response:
riik
=
l(f,(ai), f2(bi), fa(ck))
( 1)
which results in a manifest response Rij k by means of the response function M:
Ri/k = M(ri/k)
(2)
A scheme of these processes is presented in fig. 1 which is a modified version of Anderson's (1970, 1974a, 1977) functional measurement diagram. STIMULUS COMPONENTS
ai
PSYCHOLOGICAL C OSTIMULUS MPONENTS
.
~, f l i a i .) f 2 ( b j )
ck
:> f 3 ( c k )
VALUATION FUNCTIONS
MANIFEST
RESPONSE
)
bj
I
LATENT
RESPONSE
N
~- rij k
T
E
G
R
A
INTEGRATION FUN CT ION
T
O
:)' Rij k
R
RESPONSE FUNCT ION
Fig. 1. A scheme of Andersons's information integration theory (this is a modified version of Anderson's (1970, 1974a, 1977) functional m e a s u r e m e n t diagram).
G. De Soete / hfformation integration
259
Integration theorists generally take the valuation for granted and concentrate primarily on the integration. Mervielde (1977) explains this by indicating that valuation itself can be seen as a result of an integration operation. Anderson has repeatedly shown how simple algebraic models are apt to describe this process. Especially additive models were successful. Anderson draws a sharp psychological distinction between what he calls a simple adding and an initial impression weighted averaging model. In the common case of constant weights per factor, both models can respectively be written as: rii k = w l f a ( a i ) + w2f2(b/) + w 3 ~ 3 ( c k )
(3)
and
rijt = 00Io + ViA(ai) + v2f2(b,) + v3f3(cD
(4)
with v0 + vl + v2 + v3 = 1. In eq. (4) I0 stands for the initial impression of the subject. Contrary to the w in (3), the v in (4) are only relative weights. However, it can be proved (cf Sch6nemann et al. 1973) that whenever the data fit the following model: rijk = ~l(ai) + ~b2(bi) + 4~3(ck)
(S)
they also fit eqs. (3) and (4). Consequently, in this paper we do not distinguish between (3) and (4) and we consider the additive model (5) only. Although the additive model has played a dominant role, several kinds of interactive models have, rather sporadically, been found. Integration models are usually validated by means of functional measurement, a methodology developed by Anderson (1970, 1974a, 1977, 1978a) parallel to his integration theory. In most information integration experiments the overt responses consist of ratings, which are analyzed by means of ANOVA. Absence of significant interaction effects is interpreted as evidence for an additive model and for the validity of the assumption that the response scale is an interval scale (cf. Anderson 1977, 1978a; Klitzner and Anderson 1977). If however an interaction is detected, one is never sure whether this is due to the judgmental process itself, or to the nonlinearity of the response function. In this case, it is safer to assume only a monotonic relationship between the latent and manifest responses and to diagnose the model by relying
260
G. De Soete /hLtbrmation integration
on the ordinal characteristics o f the data. This can be done by axiomatic conjoint measurement that provides a num ber of qualitative tests (axioms) which are necessary and ahnost sufficient conditions for the different combination rules (Krantz e t al. t 971; Krantz and Tversky 1971). Once the underlying model is determined in this way, the data can be scaled according to it and the relation between the latent and manifest responses can be empirically investigated by means of polynomial regression.
The integration of motivational information When judging the future performance of a person, people generally have not only an impression o f the skills and abilities of the person, but they also know more or less his motivation. How will this motivational information influence the judgment? Intuitively, the idea of motivation as a multiplier seems appealing. This is for instance apparent from Hull's well known formula: Reaction Potential = Drive X H a b i t . The idea occurs also explicitly in Heider's (1958) attribution theory: Performance = Motivation X Ability (1958: 8 3 ) .
{6)
This proposition has been empirically verified by Anderson and Butzin (1974), together with two derived formulas: Motivation = Performance X (Ability) -1
(7)
Ability = Performance × (Motivation)-'
(8)
ANOVAs on the ratings o f two different groups of subjects indicated for (7) as well as for (8) rather an additive than a multiplicative model. Only eq. (6) has been confirmed. However, if one considers the averaged ratings with regard to (6), as plotted in the first panel of figs. 1 and 2 of Anderson and Butzin (1974), as ordinal data, one can represent them very well according to an additive model by lneans o f ADDALS (de
G. De Socte /Information integration
261
Leeuw et al. 1976) [11. The stress [2] value for both groups is less than 0.0001. Similar results have been obtained by Ullrich and Painter (1974). In their experiment students had to judge job applicants according to their ability for filling a managerial position, given their intelligence, experience and achievement motivation. A conjoint measurement analysis of the results revealed an additive model for several subjects. In order to investigate further the role of motivational information we performed the following experiment. University students were asked to judge the chances to pass of a set of hypothetical co-students which were characterized by their intelligence (l), motivation (M) and the extent to which they studied (S). Working with three factors instead of two has subtantial advantages (Klitzner and Anderson 1977) of which the most important is that a better discrimination between additive and interactive models is possible. Given the findings in person perception and attribution (Anderson 1974b, c, 1978b), we can expect in accordance with the results of Ullrich and Painter (1974), an additive model I + M + S. If however Anderson and Butzin's (1974) proposal Future Performance = Past Performance + Motivation × Ability (1974: 609) is valid, a dual-distributive model S + M X I is to be found because the extent to which one studies is a kind of performance, which in turn results from an interaction between ability and motivation. If on the contrary one considers S as a part of the ability to succeed, then a distributive combination rule (I + S) X M must be expected.
The e x p e r i m e n t
Method Su bjec ts E i g h t e e n first year s t u d e n t s , 7 males and 11 females, all enrolled in the F a c u l t y o f Psychological and E d u c a t i o n a l Sciences at the University o f G h e n t , p a r t i c i p a t e d [ 1 ] We thank Norman H. Anderson for making this data available to us. [2] The stress is a goodness of fit measure which is the square root of the normalized sum of squared deviations between tile optimally transformed data and the model. ADDALS uses Kruskal's (1965) stress formula two.
262
(;. De Soete /In¢brmation in tegration
v o l u n t a r i l y in partial f u l f i l m e n t s o f certain class r e q u i r e m e n t s . No o n e was a c q u a i n t e d e i t h e r with c o n j o i n t m e a s u r e m e n t or with A n d e r s o n ' s i n f o r m a t i o n integration theory. Stimuli T h e s t i m u l i c o n s i s t e d o f d e s c r i p t i o n s o f h y p o t h e t i c a l first year s t u d e n t s , w h i c h were c h a r a c t e r i z e d by t h r e e f e a t u r e s : intelligence (3 levels: IQ = 100, 120, 140), m o t i v a t i o n (3 levels: n o t , weakly, s t r o n g l y m o t i v a t e d ) a n d the e x t e n t to w h i c h t h e y s t u d i e d (2 levels: studies little, m u c h ) . By c o m b i n i n g t h e t h r e e variables factorially, 18 stimuli were o b t a i n e d . I n c l u s i o n of an a d d i t i o n a l s t i m u l u s (IQ = 125, weakly m o t i v a t e d , studies little) for c o n v e n i e n c e of design yielded a t o t a l of 19 stimuli. Procedure As we i n t e n d e d to a n a l y z e the j u d g m e n t s n o n m e t r i c a l l y ( a x i o m analyses) as well as m e t r i c a l l y ( p o l y n o m i a l regressions), it was desirable to g a t h e r n o t o n l y ratings b u t also explicit o r d i n a l data, because ties or m i n o r o r d e r inversions in the ratings due t o r a n d o m j u d g m e n t a l f l u c t u a t i o n s can cause m a n y v i o l a t i o n s against t h e c o n j o i n t m e a s u r e m e n t axioms. W h e n b o t h r a n k o r d e r data and ratings m u s t be o b t a i n e d at t h e same t i m e , it is b e t t e r to ask the s u b j e c t s t o o r d e r t h e stimuli first, o t h e r w i s e t h e r e is a big c h a n c e t h a t t h e y will use t h e i r ratings while r a n k o r d e r i n g t h e stimuli. In o r d e r to be able to i n t r o d u c e a n u m b e r of r e p l i c a t i o n s per subject t h e stimuli were a r r a n g e d in a b a l a n c e d i n c o m p l e t e b l o c k design ( C o c h r a n and Cox 1957: plan 11.32), c o m p o s e d of 19 b l o c k s of I0 stimuli. T h e subjects were r e q u e s t e d to r a n k o r d e r stimuli w i t h i n blocks. In this m a n n e r e a c h s t i m u l u s was j u d g e d ten times, while each pair of stimuli was p r e s e n t e d five times. T h e subjects were r u n in group. E a c h subject received a c o m p u t e r g e n e r a t e d b o o k l e t . On the first page t h e y were i n s t r u c t e d t h a t t h e p u r p o s e o f t h e e x p e r i m e n t was to investigate h o w well first year s t u d e n t s in p s y c h o l o g i c a l and e d u c a t i o n a l sciences were already able to p r e d i c t the success of c o - s t u d e n t s at the university. This mild d e c e p t i o n was inspired u p o n Mervielde ( 1 9 7 7 ) . T h e f o l l o w i n g pages cont a i n e d t h e 19 b l o c k s w h i c h h a d to be r a n k o r d e r e d i n d e p e n d e n t l y a c c o r d i n g t o t h e c h a n c e s t h e described s t u d e n t s h a d to pass at t h e university. T h e o r d e r of t h e b l o c k s as well as t h a t of the s~imuli w i t h i n each b l o c k was r a n d o m i z e d p e r subject. Finally t h e 19 stimuli were p r e s e n t e d again a n d the s u b j e c t s h a d to i n d i c a t e o n an eleven p o i n t scale (going f r o m 0 to 10) h o w m a n y c h a n c e s to ten the s t i m u l u s s t u d e n t s h a d to pass. T h e whole task t o o k o n t h e average 65 m i n u t e s t o c o m p l e t e . A f t e r w a r d s the subjects were asked t o fill in a q u e s t i o n n a i r e a b o u t t h e e x p e r i m e n t . R esu Its
Orderings and agreement a m o n g subjects One s u b j e c t did n o t u n d e r s t a n d t h e task p r o p e r l y . His data were discarded as t h e y were u n u s a b l e . F o r e a c h o f the r e m a i n i n g 17 s u b j e c t s the s t o c h a s t i c a l l y d o m i n a n t o r d e r i n g ( C o o m b s a n d H u a n g 1970) over the 19 stimuli was o b t a i n e d . As e a c h pair
263
G. De Soete /Information integration
of stimuli is replicated five times, a s t i m u l u s x is said to d o m i n a t e stochastically y w h e n e v e r x d o m i n a t e s y in m o r e t h a n t w o of t h e five w i t h i n - b l o c k c o m p a r i s o n s of (x, y). K e n d a l l ' s ( 1 9 5 5 ) c o e f f i c i e n t of c o n c o r d a n c e a m o n g t h e subjects is 0.753. W h e n t e s t e d against t h e null h y p o t h e s i s of n o a g r e e m e n t , this value is highly signific a n t (X 2 (18) = 230.5 2, p < 0.001). H o w e v e r , r e j e c t i o n of this h y p o t h e s i s does n o t i m p l y t h a t t h e r e are n o individual differences. Indeed, t h e pairwise Kendall ( 1 9 5 5 ) tau c o r r e l a t i o n s b e t w e e n t h e several orderings range f r o m 0 . 0 2 4 to 0.959. Conseq u e n t l y , t h e data h a d t o be a n a l y z e d individually.
Consistency and transitivity A first way to assess t h e c o n s i s t e n c y a m o n g the j u d g m e n t s of a single subject is to c o m p u t e D u r b i n ' s ( 1 9 5 1 ) test statistic w h i c h u n d e r the null h y p o t h e s i s t h a t each r a n k i n g in each b l o c k is equally likely, is a p p r o x i m a t e l y d i s t r i b u t e d as a chi-square. F o r each subject, t h e X2-value associated w i t h t h e statistic is listed in the first c o l u m n o f t a b l e 1. As t h e critical value at t h e 0.001 level is 4 2 . 3 1 , t h e null h y p o t h e sis c o u l d be r e j e c t e d for all subjects. A n o t h e r way o f l o o k i n g at t h e c o n s i s t e n c y is b y i n s p e c t i n g t h e s u b s e q u e n t choices o n e a c h pair of stimuli. C o o m b s a n d H u a n g ( 1 9 7 0 : 3 2 8 ) n o t i c e d t h a t w i t h five r e p l i c a t i o n s of a 5 0 / 5 0 choice o n each pair, the d i s t r i b u t i o n of the d o m i n a n t s t i m u l u s is a f o l d e d b i n o m i a l over 3, 4 a n d 5 w i t h a m e a n of 3.44 a n d a s t a n d a r d d e v i a t i o n of 0.371. A significant deviation f r o m c h a n c e at the 0.01 level (one-tail test) for t h e average over t h e ( 1 9 ) = 171 pairs is 3.52 or m o r e . T h e average con-
Table 1 Consistency and transistivity. Subject
x 2 associated with Durbin's statistic
Consistency
Number of circular triads
L A R Q G K J F C E P N D B I M H
162.90 161.43 162.44 161.11 157.96 156.75 155.39 159.00 151.65 153.46 152.18 153.58 150.78 151.28 150.59 143.75 136.01
4.988 4.971 4.965 4.936 4.801 4.790 4.766 4.760 4.760 4.754 4.725 4.714 4.714 4.696 4.667 4.602 4.515
0 0 0 0 1 0 2 0 2 1 4 3 4 4 0 5 7
264
G. De Soete /InJormation inte,gration
sistency per s u b j e c t is p r e s e n t e d in t h e s e c o n d c o l u m n of table 1. F o r all subjects t h e h y p o t h e s i s o f r a n d o m n e s s c o u l d be rejected. Besides e s t i m a t i n g the c o n s i s t e n c y of the data, it is also i m p o r t a n t to evaluate t h e t r a n s i t i v i t y , b e c a u s e , if t r a n s i v i t i t y does n o t hold, the overall o r d e r i n g cons t r u c t e d o u t o f the w i t h i n - b l o c k r a n k i n g s is of no m e a n i n g . T h e t r a n s i t i v i t y of t h e s t o c h a s t i c a l l y d o m i n a n t choices per pair can be assessed b y c o u n t i n g t h e n u m b e r o f circular triads ( K e n d a l l 1955). With 19 stimuli, t h e n l a x i m u m n u m b e r is 285, while the e x p e c t e d n u m b e r a s s u m i n g a 5 0 / 5 0 pairwise c h o i c e is 242. T h e n u m b e r o f circular triads for each s u b j e c t is given in c o l u m n t h r e e of table 1. As is a p p a r e n t f r o m table 1, t r a n s i t i v i t y can be said to h o l d q u i t e well for all subjects.
Independence A c c o r d i n g to K r a n t z and T v e r s k y ' s ( 1 9 7 1 ) f l o w c h a r t for t h e diagnosis of three-fact o r p o l y n o m i a l s in the u n s i g n e d case, o n e has to test for i n d e p e n d e n c e in the first place. F a c t o r l is said to be i n d e p e n d e n t of M for a fixed level of S, if t h e r a n k o r d e r o f I is t h e same at all levels of M. This can be assessed b y c o m p u t i n g K e n d a l l ' s c o e f f i c i e n t o f c o n c o r d a n c e b e t w e e n t h e r a n k orders o f I over all levels of M (Wallsten 1976). Moreover, this can be d o n e at e a c h level o f S. T h e s e c o e f f i c i e n t s revealed t h a t 14 o f t h e 17 s u b j e c t s satisfied t h e i n d e p e n d e n c e r e q u i r e m e n t perfectly. F o r one s u b j e c t , M, f a c t o r M was n o t c o m p l e t e l y i n d e p e n d e n t of I. Since the c o e f f i c i e n t of c o n c o r d a n c e , averaged over the levels of S, a m o u n t e d to 0 . 9 3 9 , o n l y a small v i o l a t i o n was involved. T h e two r e m a i n i n g subjects, D a n d H, v i o l a t e d the a x i o m m o r e seriously. T h e i r l o w e s t averaged coefficients of c o n c o r d a n c e were respectively 0 . 0 7 4 ( f o r S i n d e p e n d e n t of M) a n d 0 . 2 0 3 (for 1 i n d e p e n d e n t of M). In cases like these, o n e has to test for sign d e p e n d e n c e . This was n o t d o n e here because t h e factors h a d n o e n o u g h levels to p e r f o r m t h e o t h e r tests r e q u i r e d in t h e signed case. Double cancellation T h e d o u b l e c a n c e l l a t i o n a x i o m could o n l y be t e s t e d for t h e factors I a n d M, as m i n i m a l l y a 3 X 3 m a t r i x is required. In each test of t h e c a n c e l l a t i o n c o n d i t i o n , six cells are involved. In all t h e d i f f e r e n t tests w h i c h are possible in a given 3 X 3 m a t r i x , o n l y six d i f f e r e n t six cell c o m b i n a t i o n s are involved. "The d o u b l e c a n c e l l a t i o n c o u l d be t e s t e d at each level o f S. C o n s e q u e n t l y , 12 d i f f e r e n t six cell c o m b i n a t i o n s could violate t h e c o n d i t i o n . O n l y o n e s u b j e c t , P, failed to satisfy t h e d o u b l e c a n c e l l a t i o n in o n e single six cell c o m b i n a t i o n . All o t h e r subjects, i n c l u d i n g s u b j e c t s D a n d H, satisfied t h e a x i o m perfectly. Join t independence T h e j o i n t i n d e p e n d e n c e was verified n e x t . F a c t o r s I and M are said to be j o i n t l y i n d e p e n d e n t o f S, if t h e r a n k o r d e r over all c o m b i n a t i o n s o f / a n d M is t h e same at all levels o f S. T h e a g r e e m e n t b e t w e e n t h o s e r a n k orders can again be expressed b y K e n d a l l ' s c o e f f i c i e n t of c o n c o r d a n c e . T h e s e c o e f f i c i e n t s were c a l c u l a t e d for the d i f f e r e n t j o i n t i n d e p e n d e n c e tests. A l t h o u g h m o s t subjects s h o w e d s o m e m i n o r violations, t h e results were very similar to t h o s e of the i n d e p e n d e n c e analyses: serious v i o l a t i o n s o c c u r r e d o n l y w i t h s u b j e c t s D a n d H.
265
G. De Soete / Information integration Table 2 Goodness of fit for ADDALS scalings and results of regression analyses. Subject
Kendall tau
Stress
r2
F-ratio *
L A R Q G K J F C E P N D B I M H
1.000 1.000 0.987 1.000 0.951 0.974 0.941 0.922 0.980 0.937 0.907 0.931 0.951 0.964 0.948 0.927 0.864
0.000 0.000 0.000 0.000 0.002 0.000 0.002 0.049 0.000 0.004 0.000 0.054 0.000 0.001 0.017 0.002 0.002
0.956 0.983 0.782 0.961 0.786 0.912 0.742 0.843 0.838 0.842 0.893 0.847 0.847 0.848 0.890 0.854 0.738
347.21 967.39 57.48 349.26 58.85 166.68 46.11 85.77 82.71 85.52 134.19 88.54 88.27 89.33 130.04 93.39 45.15
* All F-ratios have one degree of freedom for the numerator and 16 for the denominator, except for subject A who forgot to rate one stimulus, so that the dffor the denominator for this subject become 15. All F's are highly significant (p < 0.001).
Model characterization F i f t e e n o f the 17 subjects satisfied t h e i n d e p e n d e n c e , double cancellation and j o i n t i n d e p e n d e n c e a x i o m s very well. C o n s e q u e n t l y , t h e i r j u d g m e n t s could best be d e s c r i b e d by t h e additive m o d e l 1 + M + S. A discussion o f t h e resurts for subjects D and H is p o s t p o n e d for a while. Because an additive c o m b i n a t i o n rule seems a p p r o p r i a t e for m o s t subjects, we have scaled each stochastically d o m i n a n t o r d e r i n g a c c o r d i n g to this m o d e l by m e a n s o f t h e A D D A L S a l g o r i t h m (de L e e u w et al. 1976) [3]. S o m e g o o d n e s s o f fit measures for the additive r e p r e s e n t a t i o n s are r e p o r t e d in table 2. The first c o l u m n conrains t h e Kendall tau correlations b e t w e e n the rank o r d e r data and t h e scale values, while the s e c o n d c o l u m n lists the stress values. The squared stress (see f o o t n o t e 2) is up t o a n o r m a l i z a t i o n f a c t o r t h e loss f u n c t i o n m i n i m i z e d b y t h e algorithm. As is evident f r o m table 2, all. r e p r e s e n t a t i o n s are q u i t e satisfactory, even for subjects D and H. The n o r m a l i z e d scale values associated w i t h the levels o f the t h r e e factors are [3] The maximum number of iterations allowed was set to 30, while the minimum stress improvement required for continuation was set to 0.0001.
G. De Soete /hz¢brmation in tegration
266
Table 3 Normalized AI)DALS scale values. Subject
I1
12
I3
S1
L A R Q G K J F C E P N D B I M tt
-0.216 -0.187 --0.760 --0.942 -0.669 -0,409 -0.369 -0.978 -0.369 -0.802 0.707 1.002 1.167 0.375 -0.914 -0.787 0.000
0.006 0.005 -0,029 0,017 0.074 0.000 0.178 0.291 0.038 0.009 -0.001 -0.097 0.045 -0.058 0.036 0.045 0.001
0.222 0.192 0.788 0.959 0.744 0.409 0.192 0.687 0.331 0.793 0.708 1.099 1.122 0.433 0.950 0.832 0.001
-0.999 --0.288 -0.909 -0,744 --0.710 -1.005 0.276 -0.548 1.009 - 0.803 0.707 -0.267 0.000 0.401 0.576 0.812 0.004
$2
0.999 0.288 0,909 0.744 0.710 1.005 0.276 0.548 1.009 0.803 0.707 0.267 0.000 0.401 0.576 0.812 -0.004
M1
M2
M3
-0.672 1.188 0.272 0.215 0,662 -0.510 -1.138 0.679 0.529 -0.290 0.706 0.508 0.409 0.983 0.413 0.256 -1.203
0.006 0.025 0.001 0.017 ---0.074 0,101 0.023 0.028 -0.121 0.250 --0.001 0.098 0.045 - 0.178 0.206 0.239 0.042
0.677 1.162 0.271 0.198 0.736 0.611 1.161 0.651 0.649 0.540 0.707 0.606 0.455 1.161 0.619 0.495 1.245
p r e s e n t e d in table 3. In o r d e r to allow for b o t h w i t h i n - s u b j e c t and b e t w e e n - s u b j e c t c o m p a r i s o n s , n o t only t h e sum o f scale values per f a c t o r but also the t o t a l sum o f s q u a r e d scale values was m a d e c o n s t a n t for each subject. By i n s p e c t i n g t h e range o f t h e scale values per f a c t o r , one can see t h a t a l t h o u g h all subjects e x c e p t D and H, t o o k a c c o u n t o f the three variables, the relative i m p o r t a n c e o f each f a c t o r varied f r o m s u b j e c t t o subject. This finding justifies our e m p h a s i s o n analyzing the data at t h e individual level. F r o m t h e scale values of subjects H and D, it can be i n f e r r e d t h a t t h e s e individuals only d i f f e r e n t i a t e d the stimuli on t h e basis o f respectively one and t w o factors ( m o t i v a t i o n and intelligence). This has given rise t o r a n d o m o r d e r inversions w h i c h caused the m a n y a x i o m violations o f these subjects. As in these cases one can hardly speak o f an imp!icit c o m b i n a t i o n rule, these subjects c a n n o t be said to disc o n f i r m a t h e o r y w h i c h p r e d i c t s an additive m o d e l .
Validation o f the rating scale The scale values o b t a i n e d by A D D A L S can be regarded as e s t i m a t e s o f the latent responses. As o u t l i n e d in the first s e c t i o n , the latent r e s p o n s e s are t r a n s f o r m e d b y the r e s p o n s e f u n c t i o n i n t o m a n i f e s t responses. If this f u n c t i o n is linear, the manifest responses, i.e., t h e ratings, can be said t o f o r m an interval scale. We have applied a stepwise p o l y n o m i a l regression p r o c e d u r e to d e t e r m i n e the shape of this f u n c t i o n . F o r all subjects a linear regression a c c o u n t e d for a s u b s t a n t i a l part o f t h e variance of the scale values, as is a p p a r e n t f r o m the s q u a r e d c o r r e l a t i o n c o e f f i c i e n t s
G. De Soete / Information integration
267
p r e s e n t e d in the t h i r d c o l u m n of table 2. T h e last c o l u m n o f t h a t t a b l e gives t h e Fr a t i o s associated w i t h t h e best linear fit. In n o case a d d i t i o n of h i g h e r o r d e r polyn o m i a l t e r m s , u p to the t h i r d degree, i m p r o v e d the fit significantly. Since t h e linearity o f t h e r e s p o n s e f u n c t i o n has b e e n assessed by this p r o c e d u r e , t h e ratings could b e c o n s i d e r e d as interval data. As a final c h e c k of t h e validity of o u r t w o m a i n c o n c l u s i o n s , viz., t h e a p p r o p r i ateness of t h e additive m o d e l and the linearity of t h e r e s p o n s e scale, a r e p e a t e d m e a s u r e s A N O V A was p e r f o r m e d on t h e ratings [4]. If b o t h o u r assertions were true, significant m a i n effects a n d n o n s i g n i f i c a n t i n t e r a c t i o n effects of t h e s t i m u l u s f a c t o r s were to be o b t a i n e d , unless of course the a s s u m p t i o n s u n d e r l y i n g the m e t h od were t o o seriously violated. T h e A N O V A results were e x a c t l y as e x p e c t e d . T h e m a i n effects of S, 34 a n d I were highly significant (resp. F ( 1 , 1 6 ) = 2 4 . 4 4 , F ( 2 , 1 6 ) = 16.54, F ( 2 , 1 6 ) = 29.91, all p < 0 . 0 0 1 ) , whereas n o n e of t h e F-values associated w i t h t h e i n t e r a c t i o n t e r m s a p p r o a c h e d significance. These results provide a d d i t i o n a l evidence for t h e validity of o u r conclusions!
Discussion and conclusions Conjoint measurement techniques have proved to be quite useful for determining the appropriate model. Although this approach lacks an adequate error theory, this did not cause any particular difficulty in interpreting the results of the axiom analyses. The judgments of all subjects who differentiated the stimuli on the basis of the three factors, could adequately be represented by an additive model. A question put forward by a referee in connection with this result concerns the power of the present 3 × 3 × 2 design to reject the additive model. Indeed, as in most conjoint measurement applications, the design was minimal, since only such a small design allows for testing the consistency and transitivity at the individual level in a meaningful way. This advantage of the 3 × 3 × 2 design is of course worthless when the design lacks enough power to validate the proposed models. Fortunately, previous experimental applications of conjoint measurement w h e r e the very same design was used show it to have the required power (cf. Coombs and Huang 1970; Ullrich and Painter 1974). Our conclusion to an additive model contrasts with the intuitive idea of motivation as a multiplier and suggests that at least Belgian students integrate motivational information in a very similar manner as other [4] The one missing rating (cf. table 2) was estimated by means of linear regression using the latent responses as obtained by ADDALS.
268
G. De Soete / b~formation integratioJt
kinds of information are processed (Anderson 1974b, c). The discrepancy between the present results and those of Anderson and Butzin (1974) may possibly be attributed to cultural differences, as a recent replication of the Anderson and Butzin study in India, conducted by Singh et al. (1979), evidences. Anderson and his colleagues have very often tried to prove the linearity of the response ftmction by heavily relying on ANOVA techniques (e.g., Anderson 1977, 1978a; Klitzner and Anderson 1977). We, on the contrary, have assessed this linearity and consequently the validity of the response scale, by relying only on the ordinal characteristics o f the data for the model diagnosis and without making preliminary distributional assumptions. This can be very useful for fttrther research in the area. Although some methodologically innovative methods have been applied, we have only been able to prove the descriptil,e validity of the additive model. Le. the model is only an as i f model. What a person really does when making a judgment that is like an addition, is not known! Much more research will be needed to answer this question!
References Anderson, N.H., 1970. Functional m e a s u r e m e n t and psychophysical j u d g m e n t . Psychological Review 7 7 , 1 5 3 - 1 7 0 . Anderson, N.H., 1974a. 'Algebraic models in perception'. 111: E.C. Carterette and M.P. Friedman (eds.), Handbook of perception. Vol. II. New York: Academic Press. pp. 2 1 5 298. Anderson, N.H., 1974b. 'Cognitive algebra: integration theory applied to social attribution'. In: L. Berkowitz (ed.), Advances in experimental social psychology. Vol. VII. New York: Academic Press. pp. 1 - 101. Anderson, N.tt., 1974c. 'Information integration theory: a brief survey'. In: D.H. Krantz, R.C. Atkinson, R.D. Luce aiad P. Suppes (eds.), C o n t e m p o r a r y developments in mathematical psychology. Vol. II. San Francisco: Freeman. pp. 236 305. Anderson, N.tt., 1977. Note on functional m e a s u r e m e n t and data analysis. Perception and psychophysics 2 1 , 2 0 1 - 2 1 5 . Anderson, N.H., 1978a. Measurement of motivation and incentive. Behavior Research Methods and I n s t r u m e n t a t i o n 1 0 , 3 6 0 375. Anderson, N.H., 1978b. 'Progress in cognitive algebra'. In: L. Berkowitz (ed.), Cognitive theories in social psychology. New York: Academic Press. pp. 103 - 1 2 6 . Anderson, N.H. and C.A. Butzin, 1974. Performance = Motivation X Ability: an integrationtheoretical analysis. Journal of Personality and Social Psychology 30, 5 9 8 - 6 0 4 . Cochran, W.G. and G.M. Cox, 1957. Experimental designs. Second edition. New York: Wiley. Coombs, C.H. and L.C. Huang, 1970. Polynomial psychophysics of risk. Journal of Mathematical Psychology 7 , 3 1 7 338.
G. De Soete /InCbrmation integration
269
de Leeuw, J., F.W. Young and Y. Takane, 1976. Additive structure in qualitative data: an alternating least squares method with optimal scaling features. Psychometrika 4 i, 471 503. Durbin, J., 1951. Incomplete blocks in ranking experiments. British Journal of Psychology (Statistical section) 4, 85-90. IIcider, F., 1958. The psychology of interpersonal relations. New York: Wiley. Kendall, M.G., 1955. Rank correlation methods. Second edition. London: Griffin. Klitzner, M.D. and N.H. Anderson, 1977. Motivation X expectancy X value: a functional rneasurement approach. Motivation and Emotion 1,347 365. Krantz, D.H. and A. Tversky, 1971. Conjoint-measurement analysis of composition rules in psychology. Psychological Review 78, 151 169. Krantz, D.H., R.D. Luce, P. Suppes and A. Tversky, 1971. Foundations of measurement. Vol. I. New York: Academic Press. Kruskal, J.B., 1965. Analysis of factorial experiments by estimating monotone transformations of the data. Journal of the Royal Statistical Society (Series B, methodological) 27, 351263. Matthatis, W., G. Ernst, U. Kleinbeck and T. Stoffcr, 1976. Funktionales Messen und kognitive Algebra. Ein methodenkritischer Beitrag zur Entscheidung zwischen Modellen der Informationsiutegration. Teil I: Uebcrlegungen zur Diagnose ener Verkntipfungsregcl aus Paarurtcilen. Archiv fiir Psychologie 128,267 291. Mervielde, I., 1977. Persoonsperccptie als informatieverwerking. Unpublished doctoral dissertation, University of Ghent, Belgium. Sch6nemann, P.H., T. Cafferty and J. Rotton, 1973. A note on additive functional measuremont. Psychological Review 80, 85 87. Singh, R., M. Gupta and A.K. Dalai, 1979. Cultural difference in attribution of performance: an integration-thcorctical analysis. Journal of Personality and Social Psychology 37, 13421351. Ullrich, J.R. and J.R. Painter, 1974. A conjoint-measurement analysis of human judgment. Organizational Behavior and Hu man Performance 12, 50-61. Wallsten, T.S., 1976. Using conjoint-measurement models to investigate a theory about probabilistic information processing. Journal of Mathematical Psychology 14, 144 185.