Response coincidence analysis: a technique for assessing individual differences in response styles

Joumal of Phonelics ( 1988) 16, 401 - 41 6

Response coincidence analysis: a technique for assessing individual differences in response styles William J. Baker, John T. Hogan and Anton J. Rozsypal Department of Linguistics, The University of Alberta, Edmonton, Alberta , Canada T6G 2£7 Received 9/h December 1986 and in revisedform 23rd Jun e 1988

A new analytical technique, response coincidence ana lysis, used in conjunction with hierarchical cluster analysis, is outlined . It provides an empirical basis for identification of groups of subjects with similar response patterns. This technique is illustrated in the context of a perceptual study of Mandarin Chinese tones in which large response variability between subjects, due to differences in linguistic background , was observed .

1. Introduction

The purpose of this paper is to present a data analytic technique applicable to the assessment of individuals' variation in response scores in a wide variety of experiments. Although this technique will be applied to data gathered from a speech recognition experiment, it can be used to analyze data from psychophysical to psycholinguistic experiments. Generally speaking, in perception and psychophysical research , methods have been developed to specify a relationship between a value from a physical stimulus dimension and a response putatively elicited from a corresponding psychological dimension in the experiencing subject. The physical variable is taken as independent, i.e., under complete control of the experimenter, while the psychological variable, the response, is dependent, i.e., elicited from the observer. If the subject is treated as passive, as simply acted upon by the stimulus, variations in his responses are then explicable in terms of variations in the stimulus array. In the context of this paradigm, relationships between physical and sensory variables were first described by the Weber fraction and Fechner's logarithmic function for "just noticeable differences" . Later, Stevens revised their results and reformulated them in terms of his " power" law. These concepts are surveyed in Chapters 2 and 3 of Kling & Riggs ( 1974). Experimental results in psychophysics and perception are typically presented as stimulus-driven functional relationships and, more specifically in speech perception research , as identification or discrimination curves. Implicit in these presentations is the view that subjects function more or less as machines which receive the stimuli as input, perform a fixed algorithmic computation of some kind on that input, and then yield an output value on a psychological scale which may then be used as input to some 0095-4470/88/040401

+

16 $03 .00/0

© 1988 Academic Press Limited

402

W. J. Baker et a l.

" higher level " processing. Thi s view of the subject as a stimulus-dri ven orga ni sm, whose input-o utput rel a tionship is of sole interest to the expe rimente r, is ba sed upo n the mechanistic thinking of 19th century physics as adop ted by ea rl y 20th century beh a viori sts. As a direct consequence of such a presupposition, the possi ble ca usative or explanatory role of subject-based variables has been either largely ignored, or si mply treated as " nuisance" parameters, to be controlled if possible (usua ll y by a priori grouping) , but rarely to be take n seriousl y as a basis for ex pl a ining th e res ults . . A noteworthy exception to this is found in studies of patho logica l gro ups as op posed to " norma ls". More recent approaches to the study of sensation and perception , most notably Gibson ( 1966, 1979, and cf. Jenkin s, 1978 , in the study of mem o ry) , have recognized the active role the subject ta kes when he tries to make sense of the expe rimental demands a nd the properties of the stimulus array . This active attempt at " making sense " of the experimental situation is inescapable, forced in this sit uation just as it is in his daily experience of his norm a l world. Helson's ( 1964) " adapta tion level theory " allows for an intern a l, i. e., subject-based , scaling of responses to stimu li a nd , lik ewise, Green & Swets ( 1966), in the context o f " signal detection theory", provide a minimal poss ibility for eva luating subj ect expectations or motivations by inco rpora ting an observer bias parameter which influences the decisio n criteri on in detect ion o r di scrimination tasks. However, if th e subject-based factors which produce potent ia ll y id iosy ncratic response pattern s a re, a priori, unknown and possibly multi-faceted, these latter limited techniques are insufficient, a nd prior gro uping cannot control for such effects. A tech nique developed by Baker & Derwing ( 1982) , discussed a lso in Woods , Fletcher & Hughes ( 1986, pp. 254- 259), for the di scovery of subject groups , based o n simil a rity of response profiles in categorizing sets of stimuli , enabled these a uthors to determine relatively homogeneou s subsets of subjects who, though cons istent within groups, differed in interesting ways between gro ups. Tn a la nguage acquisition stud y, Bake r & Derwing were then able to group subj ects, in terms of the given data , into la ng uage development "stages" rather than resorting to the ad mi tted ly inadequate but o ften used dev ice of grouping subjects by ages (Wo hlwill , 1973). Analysis of gro up characteristics in terms of both background variables and consistent response pa tterns, as shown by stimulus analysis within gro ups, provided a strong empirica l basis for statements about the acquisi tio n process. The ability of th is technique to a llow analysis of q ua litative response data and to provide a metric for comparing simi larities and differences among subj ects and stimuli is seen as lendi ng itself quite readily to studies of how subjects catego rize speech stimuli. An experimenter, e.g. , might wish to exam ine phonetic properties of la nguages spoken in countries where there is great lin guistic and dialectical diversity. Tn such a case, high subject-to-subject differences attributable to sociol ingui stic factors might be expected due to differences of norms for ph onetic categories amo ng groups in a heterogeneous speech community. In some in stances the variability of subjects' linguistic back grou nds might be controlled, a priori, by grouping the subjects acco rdin g to sociolingu istic profiles and subd ividing th em into appropriate blocks. However, subjects may, through the m anner of their responses, provide much mo re subtle a nd relevant information about th eir lingui st ic perception s th an that which might be ava ila ble independent of the expe riment itself. If su bjects are first gro uped in term s of their explicit response pattern s, it ma y then be determined empirica ll y whether o r not avai lable sociolin guistic information is consistent with or helps to explain the groups observed. The criteri a and

Response coincidence analysis

403

norms of linguistic membership vary from country to country, or from region to region in large geopolitical areas; lack of knowledge of factors such as language loyalty, diglossia , the existence of preferences relative to a "p restige " dialect, or a national language, migration within a country, or dialectical continua-all of these factors serve to increase the uncerta inty of grouping based on externa l variables. Further, lack of understand ing of the relevance of any of these factors for the specific task at hand makes the imposition of group structure on the data clearly premature and more a fact to be determined by analysis of the data themselves. Clearly, it would be useful to an experimenta l phonetician who has to contend with these factors to have a method of sorting out subjects on the basis of their internal norms or strategies when responding to experimental st imuli , a method sensitive enough to sort out subjects from different dialects of the same language. Tnstead of an a priori blocking of the subjects as a statistical control, a finer partitioning based on their actual responses in an experiment could yield much richer information for the phonetician. A description of such a clustering technique for hand ling data of thi s nature is presented below . Tt might best be seen as a technique which examines how subjects categorize stimu li , as opposed to the more traditional techniques of ana lyzing how stimuli affect subjects. 2. Outline of the clustering technique

The procedure for analyzing the data encompasses two steps. The first one consists of two stages: response coincidence analysis followed by subject cluster analysis. The purpose of th is step is to sort the subjects into groups who act on, treat, or perceive the stimu li in a similar manner. It can then be assumed that each group is composed of subjects who are using simi lar strategies or decision criteria. The interpretation of what subjects in distinct groups are doing will depend , to a large extent, on the type of experiment and the type of subjects participating in it. In the second step, clustering of stimu lu s items carried out for each subject group separately, the stimuli are grouped into classes as a function of the extent to which the subjects treat them in consistent manner. The rationale for this sequence was given by Baker & Derwing ( 1983 , p. 200) as follows: Separation into st rategy groups must logicall y take place before considering how subj ec ts treat the objects being tested si nce, if general st ra tegy differences exist, mea ning sys tematica ll y different treat ment of se ts of objects by different group s of subjects, suc h differences wou ld be obscured in data pooled across a ll subj ects. Studies which a nalyze such pooled data without first co nsidering th e possibility of subject group differences make the strong implicit ass umption that a ll subjects are, within error bounds, operating on the task in exactly th e sa me way . This is tantamount to a rguing that on ly one strategy can be employed. Clearly this is a n unwarranted a ss umpti on, especially for developmental work . What is percei ved here as the logical necessit y for first establishing subject groups based on the patterns of responses within individuals (i.e., deliberatel y igno ring, initiall y, whateve r relationships wou ld appear to exist a mon g item s across subj ec ts) precludes the use of techniques wh ich a ttempt to give precedence to th e objec ts or focu s simultaneo usly on subj ec ts and objects. Carroll & C hang's ( 1970) INDSCAL ana lys is, and la ter deve lopments such as ALSCAL (Takane, Young, & De Lecuw, 1977), focus first o n the d imensionalit y of the object space, ass ume this to be common to a ll subj ects, and then compu te weights to provide a ' best' fit for indi vid ual subjects

W. J. Baker et al.

404

relative to that space (Carrol , 1972). We see a need to allow for both quantitatively and qualitatively different spaces within subject groups. Tucker's ( 1964) three-mode factor analysis and other developments a long that line (e .g., Harshman , 1970) work from a core matrix which is simultaneously determined by both subject and object and object covariances or cross-products. (Readers wishing to pursue these issues should consult Borg ( 1981) or Shepard, Romney, & Nerlove ( 1972) for extensive discussions.)

Thus, the data analytic procedures followed here, first for the classification of subjects, then of items or stimuli, begin by developing a "dissimilarity" matrix and then subjecting that matrix to a standard hierarchical cluster a nalysis using Ward's minimum variance method (Ward, 1963) as outlined in the CLUSTAN user's manual (Wishart, 1978).

2. I. R esponse coincidence analysis The problem of developing a "distance" or " dissimilarity" metric in terms of which to compare subjects has a long history in the litera ture on " profile analysis" which attempts to compare subjects in terms of the incidence of similar responses across a profile of items. Depending on what type of research questions are asked, the experimenter has several options for scoring each subject-stimulus pair as an entry in the data matrix. If the researcher is interested in the subjects' responses to each stimulus item as being correct or incorrect with respect to a given norm, then either a one or zero may be the appropriate entry. If an open response experiment is set up, then a letter or number designating each of a number of arbitrary categories can be entered into the matrix. Finally, the most usual type of experiment for identification is a forced-choice task with a fixed number of response categories. Where the responses are quantitative, e.g., scores or other measures, this leads quite naturally to the treatment of the vectors as coordinates of points in an n-dimensional space, and the computation of a Euclidean distance metric is quite straightforward, but when the responses are qualitative or categorical, the problem is considerably more difficult. And, when attention focuses on strategies across items within a subject, we must first consider the manner in which a given subject applies similar treatments to subsets of items in the stimulus inventory. Thus, rather than beginning with an " incidence" vector, we must begin with a "coi ncidence" matrix which captures the pattern of treatments applied to sets of items. The development of such matrices, which provide the basis for comparisons between subjects, is illustrated below. Let us begin with a hypothetical data set based on a forced-choice technique which presents four subjects with five stimuli which they are required to assign to one of three mutually exclusive categories, a, b or c. The resulting data appear in the following incidence matrix : Stimulus Subject Subject Subject Subject

I

2 3 4

r

a a b b

Stimulus 2

Stimulus 3

Stimulus 4

Stimulus 5

a b a a

a b a a

b b b c

b

c c c

l

(I)

Each row is an " incidence" response vector R" for subject n, indicating how the individual sorted or categorized each stimulus. From each of these row vectors, a

Response coincidence analysis

405

"coincidence" matrix is constructed which indicates information about the pairwise treatment of all possible combinations of the stimuli within each individual. Such a matrix is constructed by first taking an "outer" or "direct" product of the response vector with itself for that subject. For instance, for Subject I , the response vector from matrix (I) is a row vector R 1 [aaabb]. Its outer product wou ld then yield:

RT® Rl

a

RJ

a

aa

aa

aa

ab

ab

a

Rl

a

aa

aa

aa

ab

ab

a

RJ

a [a a a b b]

aa

aa

aa

ab

ab

b

Rl

b

ba

ba

ba

bb

bb

RJ

b

ba

ba

ba

bb

bb

b

(2)

where the column vector RT is the transpose of the row vector R 1 • Each cell entry in matrix (2) is then recoded using a scheme such that any stimulus pair sorted by the observer into the same category will receive the designation of that category and any pair that is not sorted in the sa me way will be designated by an x . In the current example, this transformation can be achieved by an operator given by the following multiplication table:

a

b

c

a

a

X

X

b

X

b

X

c

X

X

c

(3)

Thus the coincidence matrix (2) for vector R 1 from Subject I would eventuall y be recoded as

a

a

a

X

X

a

a

a

X

X

a

a

a

X

X

X

X

X

b

b

X

X

X

b

b

(4)

and , similarly, for R 2 from Subject 2 would be obtained

a

X

X

X

X

X

b

b

b

X

X

b

b

b

X

X

b

b

b

X

X

X

X

X

c

(5)

The above matrices, (4) and (5) , are symmetric along the main diagonal, and, by the logic of the operator (3) , the original response vector is mapped along this diagonal. Thus, only the information about item groupings contained in the lower triangular portion of each matrix, below the ma in diagonal , is nonredundant. This reduces matrix (4) to

W. J. Baker et al.

406 Subject I Stimulus Stimulus Stimulus Stimulus

Stimulus I 2 3 4 5

Stimulus 2

Stimulus 3

Stimulus 4

l l

a a

a

X

X

X

X

X

X

b

Stimulus I

Stimulus 2

Stimulus 3

Stimulus 4

r

and matrix (5) to Subject 2 Stimulus Stimulus Stimulus Stimulus

2 3 4 5

X

X

b b

b

X

X

X

X

r

X

(6)

(7)

which show the extent to which various stimulus items are pooled with other items, i.e. , classes of items which are treated in a similar m a nner. Matrix elements which contain the category labels a, b and c indicate that the subject regards a given pair of physically distinct stimuli as belonging to the same category. A x indicates that two stimuli are regarded as different. These matrices thus provide some insight into the strategies being employed by each individual and they also provide an obvious basis for comparing one subject with another. After obtaining the coincidence matrices for all subjects, we are in a position to define a dissimilarity measure between all pairs of subjects. The above procedures thus permitted us to convert nominal or categorized data to squared distances expressed as rational numbers. To compute such a " distance" between each pair of subjects, the lower triangular portions of their respective " coincidence" matrices are compared, element by element. For n stimuli , there are N = n(n - 1)/ 2 entries of interest in each matrix . In order to normalize the distance measure , the count of the number of mismatches between two matrices is divided by N , to yield a " distance" or " dissimilarity " metric which ranges from zero for perfect correspondence to one for completely different pairings or category assignments. For instance, to determine the dissimilarity measure between Subjects I and 2, coincidence matrices (6) and (7) are compared entry by entry. Out of ten possible stimulus pairs, six pairs were assigned to different categories which results in a dissimilarity measure of 0.60. Applying this logic to all pairs of subject coincidence matrices in our example would yield the following logic to all pairs of subject coincidence matrices in intersubject dissimilarity matrix: Subject I Subject Subject Subject Subject

I 2 3 4

r 0~0 0.40 0.30

Subject 2

0.40 0.30

Subject 3

0.20

Subject 4

-

l

(8)

Computations carried out in this manner can be shown to be equivalent to a squared distance, in a Euclidean sense (Clifford & Stephenson, 1975, p. 66), and thus provide the appropriate input for the use of Ward ' s method for hierarchical cluster analysis (Ward , 1963). For our sample data matrix (8) , Subjects 3 and 4 are most similar in their response patterns, their squared distance being only 0.20 , while Subjects I and 2 are the most dissimilar, with the squared distance between them being 0.60.

Response coincidence analy sis

407

2.2. Cluster analysis: subject groups

As the next step, a dissimilarity matrix for all possible pairs of subjects, of the form as (8), can be submitted to a hierarchical cluster analysis program such as that described by Wishart ( 1978). This will allow the investigator to determine if there are subjects with reasonably similar response patterns who can be grouped and usefully distinguished from subjects with different response patterns. There is both a " homogeneity " requirement for grouping, and a "heterogeneity" requirement for separating groups. The latter is the usual " how many groups are there " issue for cluster analysis while the former , which is just as crucial , is often totally ignored as an issue. Both points can be addressed , as will be discussed below.

2.3. Cluster analy sis: object clusters

Following the grouping of subjects, a similar type of analysis based on the objects or items as cases is then performed for all subject groups identified in the preceding step. The purpose of this analysis is to reveal any possible patterns among the stimuli within each subject group. Here we would define the " distance" between object pairs as the number of subjects who did not place the items in the same category, divided by the total number of subjects. This is, again, a metric which ranges from zero (all subjects paired the items) to 1.0 (no subjects paired the items). Homogeneous sets of items , i.e., sets of items subjected to the same treatment by the majority of subjects in a given group , are presumably perceived in a similar manner by the subjects. These concepts will be developed further in the context of our specific example.

3. Illustrative case: cluster analysis of the Mandarin Chinese tone data

For the sake of clarity of exposition of the technique, reference will be made to an experiment previously reported by Connell, Hogan & Rozsypal (1983) in which the interaction between tone and intonation was studied. Both of these prosodic features are based on pitch variation. In the experiment, three syllables: /pi/, fpa/, and / tu /, each spoken with the four Mandarin tones , were recorded. The combination of each syllable with each of the four tones yields twelve actual words in Mandarin Chinese. The pitch contour of each word was precisely a ltered by a digital gating technique (Rozsypal, 1976) using the Alligator program (Stevenson & Stephens, 1978) so as to simulate a wide range of intonational changes on these syllables in sentence final position . Nine intonational conditions from rising, through level , to falling, were created by adjustments of the pitch contour of the original tone . Five replications of the I 08 stimuli (three monosyllables by four tones, yielding the 12 words, by nine intonation contours) provided 540 presenta tions. Twenty-eight exchange students from the People's Republic of China and Taiwan volunteered to act as subjects in an identification test. The four possible response alternatives, corresponding to the same syllable but with four different tone possibilities, were indicated on a response sheet in the form of the appropriate Chinese characters. If all of the subjects had been from a homogeneous linguistic background , the following results would have been produced . For the three words recorded with Tone I (a high , level tone) , the expected change in perception would have been to a word with Tone 4 (a falling tone) as the intonation conditions progressed from rising to falling .

408

W. J. Baker et al.

In a symmetrical way, the three words recorded with Tone 4 would have changed to words with Tone I as the pitch contour conditions changed from a falling to a rising contour. The three words with Tone 2 (a high , rising tone) would change to a word with Tone 3 (a low, falling-rising tone) as the intonation conditions changes from a rising to a falling contour. Then , again in a symmetrical manner, the three words recorded with Tone 3 would be identified as words with Tone 2 as the pitch condition shifted from a falling to a rising contour. These would constitute the expected results but, since the subjects were not sociolinguistically homogeneous, it is needless to say that the expected results were realized for only some of the subjects . Prior to the speech perception experiment, all subjects declared a knowledge of Mandarin. One reason may be that it is the official language of the People's Republic of China. Also speakers of Chinese languages will insist that they speak the same language due to the fact that they share a common writing system (Wardhaugh , 1986, p . 28) . However, it is likely that their understanding of standard Mandarin would vary according to what other Chinese language they speak or to what other Mandarin dialect region they come from. Their proficiency in comprehension would also be a function of the age at which they were exposed to the standard Mandarin dialect. It is noted by DeFrancis (1984, p. 63) that differences between the Chinese languages occur mainly at the phonological and lexical levels. Variation in the tonal system is widespread among dialects of Mandarin, such as in the Hubei and Sichuan regions (De Francis, 1984, p. 60). Twenty of the subjects, as it was subsequently discovered by the use of the coincidence analysis technique, were apparently at different levels of bilingual and bidialectal proficiency. Probably, all of the subjects could understand Mandarin when contextual and discourse information was available to them. However, when lexical items were presented in a semantically neutral context of the experiment, the only available cues to them would be purely phonetic and they could only draw on their phonological knowledge for categorization . Subjects for whom Mandarin was a second language or who spoke a dialect of Mandarin may have to draw upon the knowledge of the tone categories of their first language, which may interfere with their classification of Mandarin lexical items. For example, speakers from a Hakka language region would have six tones, with two out the six being similar to two of the four Mandarin tones (Hashimoto, 1973, p. I 04). Speakers whose first language was Wu would have a five-tone system with two tones in common with Mandarin (Zee & Maddieson , 1979). Speakers of southern Min would have a five-tone system with one tone shared with Mandarin (Yip, 1980, p. 21 0). The high-level tone was common in each case . The vowel and consonant segments used in the experiment are found in each phonological system of the languages discussed. The goal of the application of coincidence analysis in this case was to target the group of proficient Mandarin speakers and then analyse tone identification results for this group alone. Following the conventional cluster analysis technique as applied to the dissimilarity matrix derived for the 28 subjects in the Mandarin Chinese tone experiment, the issues of homogeneity and heterogeneity mentioned above had to be considered. There are no general statistical criteria which have been found to be satisfactory for resolving the heterogeneity or " number of groups" problem (cf. Seber, 1984, p. 388) because the distributional properties of distances do not lend themselves to the usual Gaussian or "normal" distribution assumptions. A moment's reflection should allow the reader to see that the distances within subject groups would be very small, ideally approaching zero. On the other hand , the distances between groups would be very large. Thus, overall, their

Response coincidence analy sis

409

distribution should be bimodal, and this would vitiate the use of the statistical methods. The problem is actually more complex than this, but it could not be discussed in detail within this paper. In spite of the lack of a general solution, however, it is possible to consider randomizations of a specific data set and to compare clusterings of randomi zed input vectors relative to the clustering derived from the structured data set. Such comparisons, for a reasonably well structured data set, would reveal that the structured set produces pairings which begin at a level much lower than that of randomized vectors , and final pairings of groups are completed at a level well above the random sets. The former point demonstrates homogeneity and justifies pooling some sets of subjects or items , while the latter justifies separating out some of the groups. As an input to coincidence and cluster analysis for the structured data set, a I x I 08 response vector was obtained from each of the 28 subjects. Each entry in this vector represents a majority response score for a particular category from five replications of a stimulus drawn from one of the 108 stimulus conditions. In order to obtain a dendrogram for a randomized data set, each of the I 08 majority response entries from a subject is randomly reassigned to an element in the response vector. For this analysis, three dendrograms based on three independent randomizations of the data were produced. Each dendrogram contained 27 bifurcations. In each tree, none of the pairings occurred below a dissimil arity value of 0.35 or above 0.75. This suggests that any pairing below the 0.3 5 level in the structured data set indicated homogeneity and thus justification for grouping of subjects. Similarly, any pairing of subject groups above 0 .75 definitely justifies separation of the groups. As will be noted in connection with the selection of the criterion for the separation of subjects into groups, only two pairings in each tree were above a value of 0.55 Hopefully, the grouping based on the randomization tests should conform to expectations of the number of groups and their composition based on the information about the subjects' background obtained in pre-experimental interviews. 3. 1. Subject groups Figure I represents the subject clustering dendrogram. On preliminary inspection of the dendrogram , a horizontal line drawn at dissimilarity coordinate of 0.55 partitions the subjects into four groups, one of which is a homogeneous group of Mandarin speakers. The criterion value of0.55 is somewhat below the upper dissimilarity limit obtained from the randomized data. However, on ly two pairings in each of the dendrograms for the randomized data occurred above this value. For ease of reference, the groups are numbered from left to right as follows : the first 10 subjects constitute Group I. Group II consists of only one element, Subject 7. Group III contains nine subjects and Group IV eight. As part of the experiment, information was gathered as to the first and second language of the subject and the province the subject lived in. This information was used to interpret the subjects' membership in the groups determined from the dendrogra m. Group I contained seven subjects who spoke Wu, a language centered around Shanghai and its province. The remaining three subjects of Group I spoke Mandarin, two from Hubei and one from Sechuan provinces which are the southern and south-western dialects of Mandarin. The single subject in Group II spoke Hakka as a first language , spoken just north of the Can tonese region. Group III is linguistically more heterogeneous in that three subjects speak Wu as a first language, two speak Xiang, i.e. Hunanese,

W. J . Baker et al.

410 1· 600 1·400 1·200

1·000 0·800 0·600 0·400 0·200 0·000

Figure I. Subject clu ster fo r th e C hin ese tone d a ta. Numbers a long the bottom id entify indi vidual subject s. Vertical scal e indica tes di ssimilarity between subjects or subject clusters.

one speaks Min from Haina n, and the remaining three speak each the Manchurian, Taiwanese, and Northwestern variety of Mandarin. Most subjects in this group appeared to have a greater familiarity with the Northern dialect of Mandarin spoken around Beijing than those in Group f. Group IV contained four monolingual Mandarin speakers, three Taiwanese Min speakers who were educated in Mandarin, and one person from Shanghai whose second language was Mandarin. This group responded to the stimuli in a way anticipated by the experimenters. The results of the identification experiment for this group were reported by Connell et a!. ( 1983). 3.2. Identification curves

Once the subject groups are separated out, separate identification curves for each group can be drawn . The four graphs of Fig. 2 represent these curves for the four tones of the stimulus fpaf. Only the responses for the original tone and the most frequent alternate were plotted. The identification results for the / tu/ and / pi / stimuli are similar. A pooled identification curve is also graphed in order to indicate the type of identification that would result if the cluster analysis were not carried out. In all four graphs the pooled data curves inaccurately represent the results of the subjects' categori zation. The most salient example appears in Fig. 2(d) where the pooled identification curve has a much more restricted range of identification values. Figure 2(d) also shows that the single person in Group II consistently sorted all nine intonation conditions into one category. This was rather unique and is indicated as such in the subject clustering. Similarities between identification curves for Groups IV and III in plots of Fig. 2 indicate that the identification pattern for Group III was much closer to that of Group IV than either of the other two groups . 3.3. ONect clusters In order to establish patterns of tone identification for subjects in each subgroup , our separate cluster analyses were carried out on the 108 stimuli sorted into the four tone

Response coincidence analysis

411

groups for the vowels /i/, jaj , a nd j uj under nine experimental conditions. As expected, the most interpretable stimulus clustering was obtained from the data of Group TV presented in Fig. 3. Size reduction of the plot required listing of the stimulus labels in the caption. To separate the stimuli into four tone groups, a horizontal line is drawn from the 6.00 point on the vertical axis which indicates the degree of dissimilarity of treatment of objects by the subjects in Group IV. That line intersects four branches of the tree structure and thus determines four highly dissimilar clusters of stimuli. The subsequent subclustering at a lower dissimilarity level indicates that the stimuli within these clusters were treated in a similar and consistent manner by the subjects of Group IV . The four clusters from left to right consist of stimuli sorted into the Tone I, Tone 4, Tone 3, and Tone 2 categories, respectively. Tn each subcluster, there is a further subdivision into two groups. One of the groups consists of stimuli that are joined at zero dissimilarity. These stimuli were classified into each tone category at or near 100% identification rate . They appear in the low flat sections of the cluster. The other branch within each subdivision of a tone group consists of stimuli that were sorted into each respective category, not at the 100% identification rate, but at one ranging from roughly 80% to 99 % . Moreover, the alternative responses were not in the expected complementary category for these data. For example, in the Tone I subcluster the alternative choices were either Tone 2 or Tone 3 but not the expected complementary Tone 4 category. The information found in the object clusters for Group IV is also reflected in the identification curves for that group. In Fig. 2(a), representing the identification results for jpa/ with Tone I, the curves are fairly flat from Condition I to Condition 7, then they jump down to I 0% recognition for Tone I and to 90% recognition for Tone 4. Fig. 2(d) shows that the eight subjects responded with greater than 90% recognition rate from Condition I to 5 as jpa/ with Tone I. From Condition 7 to 9, the subjects identified the stimuli as jpa/ with Tone 4. These two identification curves indicate why the preponderant " Tone I" respon ses and the fewer " Tone 4" responses form one large and one small cluster at a dissimilarity value of zero. A similar case holds for Tone 2 and 3 stimuli in Fig. 2(b) and (c) respectively. There are many I00% identifications for Tone 2 in each of these curves and only a few for Tone 3. The crossover point occurs at Condition 8 in Fig. 2(b) and near Condition 6 for Fig. 2(c). Thus the many Tone 2 identifications are represented by the low flat subdivision in the last subcluster and the few near I 00% identifications for Tone 3 by the small block in the second last subcluster. Figure 4 represents the object cluster for the ten subjects in Group I who mainly spoke Wu as a first language. The vertical axes of Figs 3 and 4 have the same dissimilarity scale. It is immediately evident that the dissimilarities among objects for Group I are much smaller than those for Group IV. Following the arbitrary criteria adopted for the Group IV object clusters, the reader may again draw a horizontal line from the 6.00 value on the vertical dissimilarity axis . In this case, the line will not intersect any branching lines at all. This means that in this case the subjects were not systematically partitioning the stimuli into four tone classes. The graph suggests that there may be a tendency for the subjects to form four subclusters but it is not very convincing. It can be seen from Fig. 2(a)-(d) that the identification curves for a given tone follow the same pattern in that they cross at the same category boundary. While for Group IV these curves typically span the range from I 00% to 0% identification, the curves for the remaining groups are shallower, ranging approximately from 70% to 30% or even from 60% to 40%.

W. J. Baker et al.

412

\~

(a) ~

GI Tl

A GIT4

8

•

G II Tl

o

G II T4

'f G III Tl

V G III T4 40

•

GNTI

D GIll T4 •

20

0 Pooled T4

~

"

.<::>

c;

Pooled Tl

0

(.)

;;:::

·-·-·-·-· ---.., -· ~

c


C2


"

'""

.~.,---

(b) ¢

---------.

~

:::~·-.:~.

G I T2

A GI T3 • G li T2 0 G II T3

.~

'f G

.,

m T2

A

V Gill T3

• GIll T2 D GNT3 • Pooled T2 0 Pooled T3

Cl

C2

C3

C4

C5

C6

C7

C8

C9

Intonation conditions

Figure 2. Identification curves of /pa/ with (a) Tone I and (b) Tone 2 for Groups I, II , Ill , IV and for ungrouped subjects. Identification curves of /pa/ with (c) Tone 3 and (d) Tone 4 for Groups I, II , Ill, IV and for ungrouped subjects.

The very low dissimilarity values in Fig. 4 do not represent I 00% agreement for a category but another sort of consistent behaviour among the subjects of Group I: namely, they were all sorting some of the stimuli randomly , i.e., near 25% identification rate.

413

Response coincidence analysis

100

• G IT3 1:. G I T2 80

+GIIT3 ¢

GII T2

1' G liT T3 V G liT T2

20

~ c:: 0

'5u

~ c::

"'

:Q

"'c::

~

o

;:~~~ ·---·---·---·---· .

.

.

.

DGN T2

~

.

OPoo,,dT3

?~----'--O-P_oo_le_d_T_4_. (d)

·---•---+---•---·---·---·---·---· ./

• G I T4

./\,

~· A

1:. G I Tl

+ G II T4 ¢

G II Tl

1' G 1li T4 V G liT Tl

• GIlL T4 D G Til Tl

• Pooled T41 0 Pooled Tl

Intonation condition Figure 2. Continued.

4. Conclusion For the example application discussed in this paper, the linguistic homogeneity was indicated from the questionnaires filled in by the subjects. In the face of consequent unexpected variation in categorization by the subjects, reanalysis via the above technique was applied to the data and a subset of proficient Mandarin speakers was successfully identified and their results were chosen for a subsequent analysis of tone and intonation

W. J. Baker et al.

414

CI

CII

em

Clll

Figure 3. Object clusters for eight subjects in Group IV. Ve rtica l sca le indi cates dissimilarity between stimuli or stimu lus clusters. The fo ll owing •.hree-lette r code is used to id entify the stimulu s cond iti on: vowel, tone, pitch contour condition. The stimuli contain ed in each cluster of the dendrogram a re li sted from left to right. Cl: Ill , All , Ul2 , Ul3 . 144. U45 , A46. 145, Ul l , U 17. U46, 128, A28 , Ul8 , Ul9, 112, 11 3, 114,11 5, 11 6, 141 , 142.143, A 12, A l 3, A l4, Al5 , Al6, A l7 , A41 , A43 , A43, A44, A45 , U 14, Ul5 , Ul6. U4 1, U42 , U43 , U44. C ll: 117, Al9 , 11 8, A47 , 119, 149, 146, 148 , U47 , 147, Al8 , A48 , A49. U48, U49. C lll: 129, U29, U28 , 135, A36 , U34, U35 , 136. U32, U3 1, U33 , 137, 138 , 139, A29, A37 , A38 , A39 , U36 , U37 , U38 , U39. C IV: 121 , 123, 124, 125 , 126, A21, A22 , A23 , A24, A25 , A26, A27 , A3 1, A32 , A33, A34, A35 , U21 , U22. U23 , U24, U25 , U26 , U27 , 122, 127 , 13 1, 132, 133, 134.

interaction (Connell et al. , 1983). The consistency in categorization within this target group was obviously related to the communality of shared tonal categories among their members . This technique would be useful for phoneticians researching lesser known languages with widespread sociolinguistic variability. Gando ur's ( 1983) work on subjects from different language background using dissimilarity judgements between synthesized tone patterns showed two dimensions resulted from multidimensional scaling analysis. By a following hierarchical cluster a nal ysis of the dissimilarity data, he showed that the grouping of the stimuli was a function of the subjects' phonological system. With coincidence analysis where subjects' categorization of stim uli is the so urce of data , a finer partitioning of the stimuli can be achieved first by examini ng individual subject differences and similarities, and then analyzing the phonetic data within groups of individuals who behave in similar ways. The focus of the technique then is " subject-centered ", with emphasis on individual subject differences and grouping of subjects rather than just a purely "stimulus-centered" approach where individual differences are considered as an unexplainable source of error.

Response coincidence analysis

CI

err

415

CJII

CN

Figure 4. Object cluste rs fo r ten subjec ts in Group I. Vertical sca le indi cates dissim il arit y between st imuli o r stimulus clu ste rs. The same stimulu s co nditi o n coding as in Fig. 3 is used. C l: Ill , 141 , 142, 114, U 17, U45, U46. 112, 113. 11 5, 143 , 144 , 145, 11 6, 129, 126, 127 , Al6. Al7. U28. A28, U29, A29 . 121. 123, 124. 128 , U l 8, U 19, 11 7. 11 8, 148, A 18, 11 9. 149, U48 , U49, U47. 146, A49, 147 , A47, A48 , A44 , A46. C II A ll , Al4, Al3 , Al2 , A4 1, A43. A45, A42 . A15 , U l4, UIS , U l6. U ll , U 12, U13 , U42, U41 , U43 U44. C lll: 125, 132. U3 1. A36. U32. A37 , 131 , U33 , 133. 134, 135. 136, Al9. 137, 138. 139, A3 8, A 39, U34, U35 , U36. U37, U38. U39. C IY: 122. A34. A27, A23 , A25 . A26, A32. A33 . U27, A3 1, A21, A22, A24, A35. U21, U24. U22. U23 , U25, U26.

In s umm a ry, the fo ll owing benefits of the cluster ana lysis technique in conjunction with the coincidence a nal ysis can be pointed out. Some degree of data analytic control can be mai ntained when subject homogeneity cannot be guara nteed a priori. Fine partitioning of the data allows the speech researcher to obtain more typical identification curves in the face or high subject variabil ity. This technique may be valuable to other resea rchers who are in terested in indi vidua l subject differences in respo nse strategies or fine differences in perception of subjects within a homogeneous speech community. To interpret results of such as these, phonetic invest iga tors wou ld have to develop detailed questionnaires and carry out careful post-experimenta l interviews for interpretation of their results. This technique would also be valuable in an " eco logical " app roach to d ata acquisition where strict laboratory controls cannot or should not be maint a ined. The technique is m a inl y lim ited to data obtained from categorization experiments involving open-cho ice or forced-choice tasks, or to expe riments where the response can be cla ssified as right or wro ng accord ing to so me norm.

References Baker, W. J . & Derwin g, B. L. ( 1982) Respo nse coincidence analysis as ev id ence for language acq ui sition strategies, Applied P.1y cholinguistics, 3, 193- 22 1. Borg, I. (Ed.) ( 198 1) Multidimensional data representations: 11·hen and ll'hr. Ann Arbor. Ml: Mathesi s. Carroll. J.D. & C ha ng, J . J. ( 1970) Anal ysis of individua l differences in multidimen siona l sca lin g via an N-way general iza tion of Ecka rt- Young decompositi o n, Psyclwmetrika , 35, 283-3 20. Ca rro ll , J . E. ( 1972) Indi vidu a l differences in multidim ensiona l scaling. In Multidimensional scali11g: theorv and applications in the behavioral sciences (R. Shepa rd eta/. , editors). New York: Seminar Press. C lifford. H. T. & St ephenso n, W. ( 1975) An introduction to nwnerical classificat ion. New York: Academ ic Press. Con nell , B. A., H ogan, J. T. & Rozsy pa l, A. J . ( 1983) Experimenta l evide nce of interact ion betwee n tone and into nation in Mandarin C hin ese. Jouma/ or Phonetics, II , 337- 351. De Francis. J. ( 1984) Th e Chinese language: .fc/Ct andfanta.\T. Honolu lu : Universit y o f Ha wa ii Press. Gandour, J. (1983) Ton e pe rcepti o n in Far Eas tern languages, .!oumal or Phonetics. II. 149- 175. Gibso n, J . J. ( 1966) The se11ses considered as percep tual sr stems. Boston: H oug ht on Mifllin. Gibso n, J. J . ( 1979) Th e ecological approach to Fisual perception. Bos ton: Ho ughton Mifllin. Green , D. M . & Swe ts. J. A. ( 1966) Signal detect ioutheo1y and psrclwphrsics. New York: Jo hn Wiley.

416

W. J. Baker et a!.

Harshman, R . (1970) Foundations of the PARAFAC procedure: models and conditions for an 'explanatory' multi-mode factor analysis, UCLA Working Papers in Phonetics, 16, 1- 84. Hashimoto, M. J. (1973) The Hakka dialect. A linguistics study o.fits phonology , syntax and lexicon. London: Cambridge University Press. Helson , H. (1964) Adaptation /eve/theory: an experimental and systematic approach to behavior. New York: Harper & Row. Jenkins, J. J. (1974) Remember that old theory of memory? Well, forget it. American Psychologist, 29, 785- 795. Kling, J. W. & Riggs L.A . (1974) Experimental psychology (third Edition). New York: Holt, Rinehart & Winston . Rozsypal, A. J. (1976) Digital gating of speech signals, Language and Speech, 19, 57-74. Seber, G. A. F. (1984) Multivariate observations. New York: John Wiley. Shephard, R., Romney, A. & Nerlove, S. (Eds.) (1972) Multidim ensional scaling: theory and applications in the behavioural sciences. New York: Seminar Press. Stevenson, D . C. & Stephens, R. C. ( 1978) Alligator ref erence manual, vols 1 & 2. Department of Linguistics, University of Alberta. Takane, Y. , Young, F. , & DeLeeuw, J. (1977) Nonmetric individual differences scaling: an alternating least squares method with optimal scaling features , Psychometrika, 42, 7-68 . Tucker, L. R. (1964) The extension of factor analysis to three-dimensional matrices. In Contributions to mathematical psychology, New York: Holt, Rinehart & Winston. Ward, J. H. (1963) Hierarchical grouping to optimize an objective function, Journal of the American Statistical Association, 58, 236- 244. Wardhaugh , R. (1986) An introduction to sociolinguistics. Oxford: Basil Blackwell. Wichart , D. (1978) CLUSTAN user manual (third Edition). Edinburgh : Program Library Unit, Edinburgh University. Wohlwill, J. F . (1973) Th e study a_{ behavioural de velopment . New York: Academic Press. Woods, A. , F letcher, P. & Hughes, A. (1986) Statistics in language studies. Cambridge: Cambridge University Press. Yip, M. J. W. (1980) The tonal phonology o{ chinese. Bloomington, Indiana: Indiana University Linguistics Club. Zee, E. & Maddieson , I. (1979) Tone and tone sandhi in Shanghai: phonetic evidence and phonological analysis, UCLA Working Papers in Phonetics, 45 , 93- 129.