- Email: [email protected]
Development of a rating scale to measure social skills in children: The matson evaluation of social skills with youngsters (MESSY)
Behno. Res. Ther. Vol. 21, No. 4, pp. 335-340. Printed in Great Britain
0005-7967/x3
1983
$3.00 + 0.00
Pergamon Press Ltd
DEVELOPMENT OF A RATING SCALE TO MEASURE SOCIAL SKILLS IN CHILDREN: THE MATSON EVALUATION OF SOCIAL SKILLS WITH YOUNGSTERS (MESSY) JOHNNY L, MATSON,*
ANTHONY F. ROTATORI
and WILLIAM J. HELSEL
Department of Learning, Development and Special Education, Northern Illinois University, Dekalb, IL 60115, U.S.A. (Received 25 October 1982)
Summary-The development of a scale for assessing social skills with children is reported. The Matson Evaluation of Social Skills with Youngsters (MESSY) was completed on 744 children between 4 and 18 years of age. A self-report form was completed on 422 children while 322 children were rated on a teacher-report measure using a S-point Likert-type scale. Analyses included test-retest reliability conducted at a 2-week interval using Pearson correlations, factor-analytic procedures and selected analyses of variance and appropriate post-hoc tests. Implications of present findings and directions for future research are discussed.
INTRODUCTION
Assessing and treating social-skill deficits has received increasing attention from clinical and developmental psychologists in recent years; a trend supported by the numerous studies appearing in the field (McFall, 1982). Most of the initial research has been treatment oriented and while adults have been the primary population studied, research with children has begun to emerge. This work is exemplified in a recent study by Matson, Esveldt-Dawson, Andrasik, Ollendick, Petti and Hersen (1980) and is aptly reviewed by Van Hasselt, Hersen, Whitehall and Bellack (1979). A major problem with research in this area has been that behaviors of children considered problematic have generally been based on a-priori assumptions of researchers rather than on well-delineated empi~cally-founded studies {Van Hasselt et al., 1979). This situation suggests the need for research which evaluates appropriate social behaviors of children before greater efforts at treatment are made. Unfortunately, little has been done in the way of developing appropriate measures of social behavior in children in a systematic fashion although some general scales of psychopathology give brief attention to the problem (Achenbach and Edelbrock, 1979). The purpose of the present study was a first attempt to develop a rating scale expressly for assessing social behavior of children. In addition to assessing test-retest reliability, the scale was factor analyzed on both self-report and other report forms. A measure such as this would seem to have broad applicability for early identification of social excesses and deficits and as a dependent variable in treatment-outcome research. METHOD
Subjects included 744 children attending both catholic and public schools in urban areas of Northern Illinois (e.g. Chicago, Rockford, Aurora). Two samples were drawn from the original n with 422 children being assigned to a self-report group while the remaining 322 children were rated by their teacher. The demographic composition of the self-report sample consisted of children 4-18 years of age with a mean of 11.19 yr. Two hundred and twelve males and 210 females were assessed. There were 383 whites, 33 blacks, 3 hispanics and 3 others (e.g. Eastern Asian, American Indian) in the sample. The teacher-report sample consisted of children 4-15 years of age averaging 7.19 yr. One hundred and seventy-six males and 146 females were evaluated. Of these, 207 were whites, 50 were blacks, 58 were hispanics and 7 were in the others category. *To whom all reprint requests should be addressed, RR’ 21/4-
A
335
JOHNNY L. MATSON er cd.
336
lnstrurnent
and procedure
In developing a rating scale for social skills in children, a number of factors must be considered, These include evaluating a wide range of verbal and non-verbal behaviors emphasizing interpersonal effectiveness without harm to others (Reardon, Hersen, Bellack and Foley, 1978; Rinn and Markle, 1980) providing a situation which maximizes reinforcers (Kelly, 1982) and which is situation specific (Combs and Slaby, 1977). A number of the most widely accepted assessment methods for children which incorporate the above components, and which have behaviors that could be placed under a general definition of social skills (Chittenden, 1942; Hersen and Bellack, 1977; Keller and Carlson, 1974) were reviewed, and selected items were inciuded in the initial version of the scale. A number of standardized measures were selected and then two independent raters chose behaviors from the scale which they believed to fit the defiinition of social skills. Measures that were reviewed included the Child Behavior Profile (Achenback, 1978; Achenbach and Edelbrock, 1979) Behavior Problem Checklist (Quay, 1977; Quay and Peterson, 1975) and Connor’s Hyperactivity Scale (Connors, 1969). The initial item pool for the MESSY (Matson Evaluation of Social Skills with Youngsters) consisted of 92 items administered to 422 children and to the teachers of 322 children. In the self-report condition each child was randomly selected by an independent rater who had been introduced to the child on several occasions. Once selected the child and rater left the regular classroom and went to a small conference room (620 x 620cm) set up to act as a temporary assessment area. It was furnished with a table and two chairs facing each other at opposite sides of the table. All extraneous materials were either taken out of the room or set to the far side. The independent raters then read standard instructions aloud to the chiId and asked him/her for feedback about the instructions he/she had just heard. Once an understanding of the instructions was demonstrated the independent rater began reading each item on the 5-point Likert scale aloud. The child was given ample time to respond. Teacher reports of randomly-assigned children were completed by the regular classroom teacher during his/her study period. Each child and teacher completed the forms on two different occasions under identical test conditions at a 2-week interval for the test-retest. Pearson correlations on test-retest were run for both samples on each item to establish a criterion correlation coefficient for exclusion/inclusion of items from the original pool. A Y = 0.50 value was chosen for the self-report and an Y = 0.55 for the teacher-report, thereby limiting the number of items for the factor analyses to optimal levels (Kim and Mueller, 1978). Factors of the MESSY were derived through Varimax rotation procedure with the 62-item self-report and 64-item teacher-report scales. Total and subfactor scores of the MESSY were then analyzed by demographic variable. This enabled the experimenters to evaluate potential significant relationships between the dependent and independent variables.
RESULTS To establish factors, item loadings of 0.30 were retained for all the factor-based scales. The resulting factors for the self- and teacher-report scales, the names given to the factors and the eigenvalues are presented in Tables 1 and 2. Factors I and II, Appropriate Social Skills and Inappropriate Assertiveness, have clear counterparts in both the self- and teacher-report versions. Factors III-V in the self-report are distinct from data obtained in the teacher-report form, however. These factors of the self-report scale have been labeled Impulsive/Recalcitrant, Overconfident and Jealousy/Withdrawal, respectively. A series of one-way analyses of variance (ANOVA) were also run. Separate analyses were conducted for both self- and teacher-report Total and Factor scores on Sex, Age and Race. Since a number of analyses were run the 0.001 rather than 0.01 level of significance was used to insure that changes were not random artifact. Post-hoc tests using the Neuman-Keuls were performed where appropriate and the confidence interval was set at 0.05. For self-report Total MESSY score x Sex was not significant whereas Sex x Inappropriate Social Behavior was significant (F[1,420] = 12.06, P < 0.001).
337
The MESSY
Table I. Items, item numbers (on left of item), correlation coefficients based on a Vatimax rotation (on right of item), and eigenvalues for each factor on the MESSY First-order -- Vatimax . ..- loadings on the MESSY for selt-report _.._ ..______
FurtarI. Appropriare
Social
Ski//
at people when I talk to them (IO) friends (12) I help a friend who is hurt (13) I cheer up a friend who is sad (16) I feel happy when someone else does well (20) I tell people they look “ice (23) I walk up to people and start a conversation (24) I say “thank you” and am happy when someo”e does something for me (28) I know how to make friends (31) I stick up for my friends (32) look at pwple when they are speaking (34) I share what I have with others (37) I show my feelings (40) I take care of others’ property as if it were my own (42) I call people by their names (43) I ask if I can be of help (44) I feel good if I help someone (46) I ask questions when talking with others (50) I feel sorry when hurt someone (52) I join in games with other children (55) I do “ice things for people who are “ice to me (56) I ask others how they are, what they have bee” doing etc. (59) I laugh at other people’s jokes and funny stories Eigenvalue = 10.59 factor If. Ina~~propriot@ Asserliueness (2) I threaten people or act like a bully (7) I take or use things that are not mine without permission (I 1) I slap or hit when I am angry (14) I give other children dirty looks (17) I pick out other children‘s fa”l~/mis~kes (19) break promises (21) I lie to get something I want (22) I pick on people to make them angry (29) I hurt others’ feelings on purpose (I try to make people sad) (30) I make fun of others (39) I make sounds that bother others (burping, sniffling) (41) I speak too loudly (53) I get into tights a lot (60) 1 think that winning is everything (61) 1 hurt others when teasing them (62) I want to get we” with someone who hurts me Eigenvalue = 4.23 (9)
I look
I havemany
I
I
I
Factor
Ill.
I become
angry
I
Faclor
I I
(0.44) (0.43) (0.56) (0.52) (0.46) (0.38) (0.39) (0.58) (0.64) (0.42) (0.50) (0.37) (0.59) (0.47) (0.42)
(0.43) (0.47) (0.54) w41 (0.35) (0.42) (0.53) (0.58) (0.52) (0.64) (0.50) (0.36) (0.55) (0.33) (0.48) (0.43)
ImpulsivejRecalcitrPnr
easily (4) I am bossy (tell people what to do instead of asking) (5) I gripe 01 complain often (6) I speak (break in) when someone else is speaking (35) I am stubborn Eigenvalue = I.91 Ficror IV. ouerco~~~~~~ (8) I brag about myself (33) I think know it all (36) I act like I am better than other people (57) 1 stay with others too long (wear out my welcome) (58) I explain things more than I need to Eigenvalue = I.18 (3)
(0.49) (0.36) (0.56) (0.57) (0.46) (0.54) (0.36) (0.57)
V. JealousyI
(0.31) (0.46) (0.50) (0.39) (0.49)
(0.30) (0.35) (0.35) (0.52) (0.47)
Withdrawal
(15) feel angry or jealous when someone else does well (38) think people are picking on me when fhey are not (49) I feel lonely (54) 1 am jealous of other people Eigenvalue = I .09 Misce/laneo~ Ilems (I) I make other people laugh (18) I always want to be first (25) I like to be alone (26) I am afraid to speak to people (27) I keep secrets well (45) I try to be better than everyone (47) I see my friends often (48) I play alone (51) I like to be the leader
(0.50) (0.43) (0.46) (0.48)
(0.27) (0.50) (0.55) (0.25) (0.13) (0.45) (0.28) (0.65) (0.61)
Social Behavior Age* was significant on Factor I of the self-report, Appropriate @‘[IO,41I] = 4.58, P c 0.001). Significant differences were evident between lo-yr olds compared to all other age groups (4-6-yr olds, 7-yr olds, B-yr olds, 9-yr olds, 1 1-yr olds, 12-yr olds, 13-yr olds, 14-yr olds, 15-yr olds and 16-18-yr olds). On the teacher-report Age x Total score was significant (F[5,316] = 12.02, P -c0.001). Neuman-Keuls posl-hoc analyses showed differences between 8-yr olds compared to 4- and 5-yr *Age groups
are ordered
from the highest
to the lowest score in each case.
JOHI+NY
338
Table
2. Items,
rotation
L. MATSON ef al.
numbers (on left of item), correlation coefficients based on a Varimax (on right of item) and eigenvalues for each factor on the MESSY
item
First-order
Varimax loadings on the MESSY ..~~ for teacher-report Fm’lor I. lnappropriare Assertiumrss~ilmpulsiveness (2) Threatens people or acts like a bully (3) Becomes angry easily (4) Is bossy (tells people what to do instead of asking) (5) Gripes or complains often (6) Speaks (breaks m) when someone else IS speakmg (7) Takes or uses things that are not his/hers without permission (8) Brags about ~m~lf/her~If (9) Slaps or hits when angry (I I) Gives other children dirty looks (12) Feels “ngry or JdOUS when someone else does well fl3) Picks out other children’s ~oits~mistakes (14) Always wants to be first (15) Breaks promises (16) Lies to get what he/she wants (17) Rcks on people to make them angry (21) Hurts others‘ feelings on purpose (tries to make people sad) (22) Is a sore lose1 (23) Makes fun of others (24) Blames othera for own problems (27) Thinks he/she knows it all (29) Is stubborn (30) Acts like he/she is better than others (31) Shows feehngs (32) Thinks people ue picking on him/her when they are not (35) Mnkes sounds that bother others (burping. sniffling) (36) Brags too much when he/she wms (381 Speaks too loudly (42) Defends self (431 Always thinks something bad is going to happat Tries to be better than everyone (48) Gets upset when he/she has to wait for things (49) Likes to be the leader (52) Gets into fights B lot (53) Is jealous of other people (SS) Tries to get others to do what he/she wants (57) Stays wth others too long (wears out welcome) (58) Explams things more than needs to (60) Hurts others to get what he/she Wants (61) Talks a lot about problems or ~orrvzs (62) Thinks that wmning 1s everything (63) Hurts others when teasing them (64) Wants to get even with someone who hurts them Eigenvalue = 26.19 Frrcror I/. doornoriare Social Skilb (I) Makes other people laugh (tells jokes, fund; s&s etc.) (IO) Helps 8 friend who is hurt (18) Walks up to people and starts it conversation for him/her (19) Says -‘thank you” and is happy when someone does somf (25) Sticks up for friends (26) Looks at people when they are speaking (28) Smiles at people he/she knows (33) Thinks good things are going to happen (34) Works well on a team (37) Takes care of others‘ property as rf it were his/her owe (39) Calls people by their names (40) Asks if he/she can be of help (41) Feels good if he/she helps others (45) Asks questions when talking with others (47) Feels sorry when he/she hurts others (SO) Joins m eames with other chddren (51) Plays by-the rules of a game (541 Does ntce thines for others who are nice to himiher i56j Asks others h& they are, what they have been doing etc. (59) Is friendly to new people he/she meets Eigenvalue = 8.25
(44)
:thing
(20) Is afraid to speak (46) Feels lonely
(0.82) (0.80) (0.84) (0.85) (0.66) (0.56) (0.65) (0.83) (0.84) (0.78) (0.79) (0.65) (0.49) (0.69) (0.87) (0.82) (0.87) (0.85) (0.78) (0.75) (0.72) (0.73) (0.55) (0.74) (0.62) (0.77) (0.68) (053)
(0.45) (0.67) (0.62) (0.57) (0.87) (0.84) (0.64) (0.72) (0.62) (0 85) (0.57) (0.78) (0.82) (0.83)
(0.35) (0.691 jO.63) (0.73) (0.72) (0.69) (0.67) (0.54) (0.59) (0.44) (0.64) (0.751 (0 66) (0.65) (0.51) (0.59) (0.44) (0.72) (0.67) (0.72)
to people
olds, 7-yr olds, 9-yr olds and lo- and 1.5-yr olds. Differences between 6-yr olds were also significant compared to 4- and 5-yr olds, 7-yr olds, 9-yr olds and IO- and 1%yr olds. Significant effects on Factor I of the teacher-report scale Inappropriate Social Behavior (F[5,316] = 12.53, P -cO.OOl), and Factor II Appropriate Social Behaviors (F[5,316] = 12.83, P < 0.001) were also noted. Differences by age group for Factor I were 8-yr olds by 4- and 5-yr olds, 6-yr olds, 7-yr olds, 9-yr olds and lo- and 15yr olds; 6-yr olds differed from 4- and 5-yr olds, 7-yr olds, 9-yr olds and lo- and 1%yr olds. On Factor II 4- and 5-yr aids differed from all other groups. Six-yr olds differed from 8- and 9-yr olds.
The MESSY
339
The current study was primarily aimed at developing a self- and teacher-report measure of social skills. Items in the final scale were established as being reliable using test-retest. Similarly, factors were established for both forms that showed significant differences for Age x Sex on the self-report subfactor of Inappropriate Social Behavior. This finding seems to follow diagnostic patterns which show that aggression and other overt inappropriate social behaviors were far higher among boys. Most prevalent studies seem to indicate that boys with emotional problems tend to display their problems more frequently on these types of behaviors than girls and varied little with age (Gersten, Langner, Eisenberg, Semcha-Fogan and McCartney, 1976; Schultz, Salvia and Feinn, 1973; Stone, Wilson, Spence and Gibson, 1969; Werry and Hawthorne, 1974; Werry and Quay, 1971). Similarly, as might be expected, scores on factors and subfactors of both the self- and teacher-report were significant. Given the wide age range and number of social-skill items, it is not unusual that differences such as this would arise. Further research to increase the number of subjects by age groups so that factor-analytic research might be done on the dimension also seems warranted and would conform to studies with children designed to assess general psychopathology (Achenbach, 1978; Achenbach and Edelbrock, 1979). Treatment studies on social skills with children and adults far that matter have primarily relied on role-play scene tests and operationally-defined behaviors The validity of role-play scenes have recently been brought into question (Morrison and Bellack, 198 1; Van Hasselt, Hersen and B&lack, t 98 1) and the use of a few operationally-de~ned behaviors as the only criteria for assessing efficacy of the complex of behaviors constituting social skills seems inadequate (Matson and Senatore, 198 I). Thus, multimode1 assessment strategies seem to be required and rating scales of a wide range of social behavior such as the MESSY could play an important role. Similarly, the selection of behaviors for treatment could be done in a more systematic fashion if methods such as normal rating scales and social-validation criteria are used to selected responses for treatment. Considerably more emphasis on research of this type by behavior therapists is warranted (Kazdin, Matson and Esveldt-Dawson, 1981).
REFERENCES Achenbach T. M. (1978) The Child Behavior Profile: I. Boys aged 6 through Il. J. cunstd~C&L Psychof,46, 478-488. Achenbach T. M. and Edelbrock C, S. (1979) The Child Behavior Profile: II. Boys aged 12-16 and girls aged 6-i I and 12-i& 2. C@tr.%&. &?I. Psj~&t?U~. 47, 223-233. Chittenden G. F. (1942) An experimenta study in measuring and rno~fyj~~ assertive behavior in young children. Moiqr. Sac. Rex Child Bev. 7, l-87. Combs M. L. and Slaby D. A. (1977) Social skills training with children. In Advances in Cfinieaf Chifd ~sycho~~~~, Vol. 1 (Edited by Lahey 3. B. and Kazdin A. E.). Plenum Press, New York. Connors C. A. (1969) A teacher rattng scale for use in drug studies with children. Am. J. Psychiar. 126, 152-156. Gersten J. C., Langner T. S., Eisenberg J. G., Simcha-Fagan 0. and McCartney E. D. (1976) Stability and change in types of behavioral disturbance of children and adolescents. J. abnorm. Child Psychol. 4, 1l-127. Hersen M. and Bellack A. S. (1977) Assessment of social skills. In Handbookfbr Behavior Assessmenr (Edited by Ceminero A. R., Calhoun K. S. and Adams H. E.). Wiley, New York. Kazdin A. E., Matson J. C. and &veldt-Dawson K. (1981) Social skill performance among normal and psychiatric inpatient children as a function of assessment conditions. B&au. Res. Ther. 19, 145-152. Keller M. F. and Carlson P. M. (1974) The use of symbolic modeling to promote social skills in preschool children with low levels of social responsiveness. Child Deu. 45, 912-919. Matson 3. L. and Senatore V. (t&l) A comp&ison of traditional psychotherapy and social sk& training fo; improving interpersonat functioning of mentally retarded adults. &&ru. Ther, 12, 369-382. Matson J. L., Esveldt-Dawson K., Andrasik F., Uhendick T. H., Petti T. and Hersen M. (1980) Direct, observational and generalization effects of social ski& training with emotionaily disturbed children. Behav. Ther. 11, 522-531. McFall R. M. (1982) A review and reformulation of the concept of sacial skills. Behav. Assess. 4, l-33. Morrison R. L. and Bellack A. S. (1981) The role of social perception in social skill. Behav. Ther. 12, 69-79. Quay H. C. (1977) Measuring dimensions of deviant behavior: the behavior problem checklist. J. abnorm. Child Psychol, 5, 177-189.
Quay H. C. and Peterson D. R. (1975) Manual for the behavior problem checklist. Unpublished manuscript. Reardon R. C.. Hersen M.. Be&k A. S. and Folev J. M. (1978) Measurinn social skill in made school bovs. d Unaublished L manuscript, University of Pittsburgh, Pennsylvania. . ’ Rinn R. C. and Markle A. (1980) Modification of skill deficits in children. In Re.rearch and Pracrice in Social Skills Training (Edited by Bellack A. S. and Hersen M.). Plenum Press, New York.
340
JOHNNY L. MATSON ef a/.
Schultz E. W., Salvia J. and Feinn J. (1973) Deviant behaviors in rural elementary school children. J. abnorm. Child Psychol. 4, 378-389. Stone F. B., Wilson M. A., Spence M. E. and Gibson R. C. (1969) A survey of elementary school children’s behavior problems. Am. J. Orthopsychiat. 39, 289-290. Van Hasselt V. B., Hersen M., Whitehall M. B. and Bellack A. S. (1979) Social skills assessment and training for children: an evaluative review. Behau. Res. Ther. 17, 413-437. Van Hasselt V. B., Hersen M. and Bellack A. S. (1981) The validity of role play tests for assessing social skills in children. Behav. Ther. 12, 2022216. Werry J. S. and Hawthorne D. (1976) Connor’s teachers questionnaire--norms and validity. Au&. N.Z. J. Psych&. 10, 257-262. Werry J. S. and Quay H. C. (1971) The prevalence of behavior symptoms in younger elementary school children. Am. J. Orih~p.~ych~al. 41. 136 143.
0005-7967/x3
1983
$3.00 + 0.00
Pergamon Press Ltd
DEVELOPMENT OF A RATING SCALE TO MEASURE SOCIAL SKILLS IN CHILDREN: THE MATSON EVALUATION OF SOCIAL SKILLS WITH YOUNGSTERS (MESSY) JOHNNY L, MATSON,*
ANTHONY F. ROTATORI
and WILLIAM J. HELSEL
Department of Learning, Development and Special Education, Northern Illinois University, Dekalb, IL 60115, U.S.A. (Received 25 October 1982)
Summary-The development of a scale for assessing social skills with children is reported. The Matson Evaluation of Social Skills with Youngsters (MESSY) was completed on 744 children between 4 and 18 years of age. A self-report form was completed on 422 children while 322 children were rated on a teacher-report measure using a S-point Likert-type scale. Analyses included test-retest reliability conducted at a 2-week interval using Pearson correlations, factor-analytic procedures and selected analyses of variance and appropriate post-hoc tests. Implications of present findings and directions for future research are discussed.
INTRODUCTION
Assessing and treating social-skill deficits has received increasing attention from clinical and developmental psychologists in recent years; a trend supported by the numerous studies appearing in the field (McFall, 1982). Most of the initial research has been treatment oriented and while adults have been the primary population studied, research with children has begun to emerge. This work is exemplified in a recent study by Matson, Esveldt-Dawson, Andrasik, Ollendick, Petti and Hersen (1980) and is aptly reviewed by Van Hasselt, Hersen, Whitehall and Bellack (1979). A major problem with research in this area has been that behaviors of children considered problematic have generally been based on a-priori assumptions of researchers rather than on well-delineated empi~cally-founded studies {Van Hasselt et al., 1979). This situation suggests the need for research which evaluates appropriate social behaviors of children before greater efforts at treatment are made. Unfortunately, little has been done in the way of developing appropriate measures of social behavior in children in a systematic fashion although some general scales of psychopathology give brief attention to the problem (Achenbach and Edelbrock, 1979). The purpose of the present study was a first attempt to develop a rating scale expressly for assessing social behavior of children. In addition to assessing test-retest reliability, the scale was factor analyzed on both self-report and other report forms. A measure such as this would seem to have broad applicability for early identification of social excesses and deficits and as a dependent variable in treatment-outcome research. METHOD
Subjects included 744 children attending both catholic and public schools in urban areas of Northern Illinois (e.g. Chicago, Rockford, Aurora). Two samples were drawn from the original n with 422 children being assigned to a self-report group while the remaining 322 children were rated by their teacher. The demographic composition of the self-report sample consisted of children 4-18 years of age with a mean of 11.19 yr. Two hundred and twelve males and 210 females were assessed. There were 383 whites, 33 blacks, 3 hispanics and 3 others (e.g. Eastern Asian, American Indian) in the sample. The teacher-report sample consisted of children 4-15 years of age averaging 7.19 yr. One hundred and seventy-six males and 146 females were evaluated. Of these, 207 were whites, 50 were blacks, 58 were hispanics and 7 were in the others category. *To whom all reprint requests should be addressed, RR’ 21/4-
A
335
JOHNNY L. MATSON er cd.
336
lnstrurnent
and procedure
In developing a rating scale for social skills in children, a number of factors must be considered, These include evaluating a wide range of verbal and non-verbal behaviors emphasizing interpersonal effectiveness without harm to others (Reardon, Hersen, Bellack and Foley, 1978; Rinn and Markle, 1980) providing a situation which maximizes reinforcers (Kelly, 1982) and which is situation specific (Combs and Slaby, 1977). A number of the most widely accepted assessment methods for children which incorporate the above components, and which have behaviors that could be placed under a general definition of social skills (Chittenden, 1942; Hersen and Bellack, 1977; Keller and Carlson, 1974) were reviewed, and selected items were inciuded in the initial version of the scale. A number of standardized measures were selected and then two independent raters chose behaviors from the scale which they believed to fit the defiinition of social skills. Measures that were reviewed included the Child Behavior Profile (Achenback, 1978; Achenbach and Edelbrock, 1979) Behavior Problem Checklist (Quay, 1977; Quay and Peterson, 1975) and Connor’s Hyperactivity Scale (Connors, 1969). The initial item pool for the MESSY (Matson Evaluation of Social Skills with Youngsters) consisted of 92 items administered to 422 children and to the teachers of 322 children. In the self-report condition each child was randomly selected by an independent rater who had been introduced to the child on several occasions. Once selected the child and rater left the regular classroom and went to a small conference room (620 x 620cm) set up to act as a temporary assessment area. It was furnished with a table and two chairs facing each other at opposite sides of the table. All extraneous materials were either taken out of the room or set to the far side. The independent raters then read standard instructions aloud to the chiId and asked him/her for feedback about the instructions he/she had just heard. Once an understanding of the instructions was demonstrated the independent rater began reading each item on the 5-point Likert scale aloud. The child was given ample time to respond. Teacher reports of randomly-assigned children were completed by the regular classroom teacher during his/her study period. Each child and teacher completed the forms on two different occasions under identical test conditions at a 2-week interval for the test-retest. Pearson correlations on test-retest were run for both samples on each item to establish a criterion correlation coefficient for exclusion/inclusion of items from the original pool. A Y = 0.50 value was chosen for the self-report and an Y = 0.55 for the teacher-report, thereby limiting the number of items for the factor analyses to optimal levels (Kim and Mueller, 1978). Factors of the MESSY were derived through Varimax rotation procedure with the 62-item self-report and 64-item teacher-report scales. Total and subfactor scores of the MESSY were then analyzed by demographic variable. This enabled the experimenters to evaluate potential significant relationships between the dependent and independent variables.
RESULTS To establish factors, item loadings of 0.30 were retained for all the factor-based scales. The resulting factors for the self- and teacher-report scales, the names given to the factors and the eigenvalues are presented in Tables 1 and 2. Factors I and II, Appropriate Social Skills and Inappropriate Assertiveness, have clear counterparts in both the self- and teacher-report versions. Factors III-V in the self-report are distinct from data obtained in the teacher-report form, however. These factors of the self-report scale have been labeled Impulsive/Recalcitrant, Overconfident and Jealousy/Withdrawal, respectively. A series of one-way analyses of variance (ANOVA) were also run. Separate analyses were conducted for both self- and teacher-report Total and Factor scores on Sex, Age and Race. Since a number of analyses were run the 0.001 rather than 0.01 level of significance was used to insure that changes were not random artifact. Post-hoc tests using the Neuman-Keuls were performed where appropriate and the confidence interval was set at 0.05. For self-report Total MESSY score x Sex was not significant whereas Sex x Inappropriate Social Behavior was significant (F[1,420] = 12.06, P < 0.001).
337
The MESSY
Table I. Items, item numbers (on left of item), correlation coefficients based on a Vatimax rotation (on right of item), and eigenvalues for each factor on the MESSY First-order -- Vatimax . ..- loadings on the MESSY for selt-report _.._ ..______
FurtarI. Appropriare
Social
Ski//
at people when I talk to them (IO) friends (12) I help a friend who is hurt (13) I cheer up a friend who is sad (16) I feel happy when someone else does well (20) I tell people they look “ice (23) I walk up to people and start a conversation (24) I say “thank you” and am happy when someo”e does something for me (28) I know how to make friends (31) I stick up for my friends (32) look at pwple when they are speaking (34) I share what I have with others (37) I show my feelings (40) I take care of others’ property as if it were my own (42) I call people by their names (43) I ask if I can be of help (44) I feel good if I help someone (46) I ask questions when talking with others (50) I feel sorry when hurt someone (52) I join in games with other children (55) I do “ice things for people who are “ice to me (56) I ask others how they are, what they have bee” doing etc. (59) I laugh at other people’s jokes and funny stories Eigenvalue = 10.59 factor If. Ina~~propriot@ Asserliueness (2) I threaten people or act like a bully (7) I take or use things that are not mine without permission (I 1) I slap or hit when I am angry (14) I give other children dirty looks (17) I pick out other children‘s fa”l~/mis~kes (19) break promises (21) I lie to get something I want (22) I pick on people to make them angry (29) I hurt others’ feelings on purpose (I try to make people sad) (30) I make fun of others (39) I make sounds that bother others (burping, sniffling) (41) I speak too loudly (53) I get into tights a lot (60) 1 think that winning is everything (61) 1 hurt others when teasing them (62) I want to get we” with someone who hurts me Eigenvalue = 4.23 (9)
I look
I havemany
I
I
I
Factor
Ill.
I become
angry
I
Faclor
I I
(0.44) (0.43) (0.56) (0.52) (0.46) (0.38) (0.39) (0.58) (0.64) (0.42) (0.50) (0.37) (0.59) (0.47) (0.42)
(0.43) (0.47) (0.54) w41 (0.35) (0.42) (0.53) (0.58) (0.52) (0.64) (0.50) (0.36) (0.55) (0.33) (0.48) (0.43)
ImpulsivejRecalcitrPnr
easily (4) I am bossy (tell people what to do instead of asking) (5) I gripe 01 complain often (6) I speak (break in) when someone else is speaking (35) I am stubborn Eigenvalue = I.91 Ficror IV. ouerco~~~~~~ (8) I brag about myself (33) I think know it all (36) I act like I am better than other people (57) 1 stay with others too long (wear out my welcome) (58) I explain things more than I need to Eigenvalue = I.18 (3)
(0.49) (0.36) (0.56) (0.57) (0.46) (0.54) (0.36) (0.57)
V. JealousyI
(0.31) (0.46) (0.50) (0.39) (0.49)
(0.30) (0.35) (0.35) (0.52) (0.47)
Withdrawal
(15) feel angry or jealous when someone else does well (38) think people are picking on me when fhey are not (49) I feel lonely (54) 1 am jealous of other people Eigenvalue = I .09 Misce/laneo~ Ilems (I) I make other people laugh (18) I always want to be first (25) I like to be alone (26) I am afraid to speak to people (27) I keep secrets well (45) I try to be better than everyone (47) I see my friends often (48) I play alone (51) I like to be the leader
(0.50) (0.43) (0.46) (0.48)
(0.27) (0.50) (0.55) (0.25) (0.13) (0.45) (0.28) (0.65) (0.61)
Social Behavior Age* was significant on Factor I of the self-report, Appropriate @‘[IO,41I] = 4.58, P c 0.001). Significant differences were evident between lo-yr olds compared to all other age groups (4-6-yr olds, 7-yr olds, B-yr olds, 9-yr olds, 1 1-yr olds, 12-yr olds, 13-yr olds, 14-yr olds, 15-yr olds and 16-18-yr olds). On the teacher-report Age x Total score was significant (F[5,316] = 12.02, P -c0.001). Neuman-Keuls posl-hoc analyses showed differences between 8-yr olds compared to 4- and 5-yr *Age groups
are ordered
from the highest
to the lowest score in each case.
JOHI+NY
338
Table
2. Items,
rotation
L. MATSON ef al.
numbers (on left of item), correlation coefficients based on a Varimax (on right of item) and eigenvalues for each factor on the MESSY
item
First-order
Varimax loadings on the MESSY ..~~ for teacher-report Fm’lor I. lnappropriare Assertiumrss~ilmpulsiveness (2) Threatens people or acts like a bully (3) Becomes angry easily (4) Is bossy (tells people what to do instead of asking) (5) Gripes or complains often (6) Speaks (breaks m) when someone else IS speakmg (7) Takes or uses things that are not his/hers without permission (8) Brags about ~m~lf/her~If (9) Slaps or hits when angry (I I) Gives other children dirty looks (12) Feels “ngry or JdOUS when someone else does well fl3) Picks out other children’s ~oits~mistakes (14) Always wants to be first (15) Breaks promises (16) Lies to get what he/she wants (17) Rcks on people to make them angry (21) Hurts others‘ feelings on purpose (tries to make people sad) (22) Is a sore lose1 (23) Makes fun of others (24) Blames othera for own problems (27) Thinks he/she knows it all (29) Is stubborn (30) Acts like he/she is better than others (31) Shows feehngs (32) Thinks people ue picking on him/her when they are not (35) Mnkes sounds that bother others (burping. sniffling) (36) Brags too much when he/she wms (381 Speaks too loudly (42) Defends self (431 Always thinks something bad is going to happat Tries to be better than everyone (48) Gets upset when he/she has to wait for things (49) Likes to be the leader (52) Gets into fights B lot (53) Is jealous of other people (SS) Tries to get others to do what he/she wants (57) Stays wth others too long (wears out welcome) (58) Explams things more than needs to (60) Hurts others to get what he/she Wants (61) Talks a lot about problems or ~orrvzs (62) Thinks that wmning 1s everything (63) Hurts others when teasing them (64) Wants to get even with someone who hurts them Eigenvalue = 26.19 Frrcror I/. doornoriare Social Skilb (I) Makes other people laugh (tells jokes, fund; s&s etc.) (IO) Helps 8 friend who is hurt (18) Walks up to people and starts it conversation for him/her (19) Says -‘thank you” and is happy when someone does somf (25) Sticks up for friends (26) Looks at people when they are speaking (28) Smiles at people he/she knows (33) Thinks good things are going to happen (34) Works well on a team (37) Takes care of others‘ property as rf it were his/her owe (39) Calls people by their names (40) Asks if he/she can be of help (41) Feels good if he/she helps others (45) Asks questions when talking with others (47) Feels sorry when he/she hurts others (SO) Joins m eames with other chddren (51) Plays by-the rules of a game (541 Does ntce thines for others who are nice to himiher i56j Asks others h& they are, what they have been doing etc. (59) Is friendly to new people he/she meets Eigenvalue = 8.25
(44)
:thing
(20) Is afraid to speak (46) Feels lonely
(0.82) (0.80) (0.84) (0.85) (0.66) (0.56) (0.65) (0.83) (0.84) (0.78) (0.79) (0.65) (0.49) (0.69) (0.87) (0.82) (0.87) (0.85) (0.78) (0.75) (0.72) (0.73) (0.55) (0.74) (0.62) (0.77) (0.68) (053)
(0.45) (0.67) (0.62) (0.57) (0.87) (0.84) (0.64) (0.72) (0.62) (0 85) (0.57) (0.78) (0.82) (0.83)
(0.35) (0.691 jO.63) (0.73) (0.72) (0.69) (0.67) (0.54) (0.59) (0.44) (0.64) (0.751 (0 66) (0.65) (0.51) (0.59) (0.44) (0.72) (0.67) (0.72)
to people
olds, 7-yr olds, 9-yr olds and lo- and 1.5-yr olds. Differences between 6-yr olds were also significant compared to 4- and 5-yr olds, 7-yr olds, 9-yr olds and IO- and 1%yr olds. Significant effects on Factor I of the teacher-report scale Inappropriate Social Behavior (F[5,316] = 12.53, P -cO.OOl), and Factor II Appropriate Social Behaviors (F[5,316] = 12.83, P < 0.001) were also noted. Differences by age group for Factor I were 8-yr olds by 4- and 5-yr olds, 6-yr olds, 7-yr olds, 9-yr olds and lo- and 15yr olds; 6-yr olds differed from 4- and 5-yr olds, 7-yr olds, 9-yr olds and lo- and 1%yr olds. On Factor II 4- and 5-yr aids differed from all other groups. Six-yr olds differed from 8- and 9-yr olds.
The MESSY
339
The current study was primarily aimed at developing a self- and teacher-report measure of social skills. Items in the final scale were established as being reliable using test-retest. Similarly, factors were established for both forms that showed significant differences for Age x Sex on the self-report subfactor of Inappropriate Social Behavior. This finding seems to follow diagnostic patterns which show that aggression and other overt inappropriate social behaviors were far higher among boys. Most prevalent studies seem to indicate that boys with emotional problems tend to display their problems more frequently on these types of behaviors than girls and varied little with age (Gersten, Langner, Eisenberg, Semcha-Fogan and McCartney, 1976; Schultz, Salvia and Feinn, 1973; Stone, Wilson, Spence and Gibson, 1969; Werry and Hawthorne, 1974; Werry and Quay, 1971). Similarly, as might be expected, scores on factors and subfactors of both the self- and teacher-report were significant. Given the wide age range and number of social-skill items, it is not unusual that differences such as this would arise. Further research to increase the number of subjects by age groups so that factor-analytic research might be done on the dimension also seems warranted and would conform to studies with children designed to assess general psychopathology (Achenbach, 1978; Achenbach and Edelbrock, 1979). Treatment studies on social skills with children and adults far that matter have primarily relied on role-play scene tests and operationally-defined behaviors The validity of role-play scenes have recently been brought into question (Morrison and Bellack, 198 1; Van Hasselt, Hersen and B&lack, t 98 1) and the use of a few operationally-de~ned behaviors as the only criteria for assessing efficacy of the complex of behaviors constituting social skills seems inadequate (Matson and Senatore, 198 I). Thus, multimode1 assessment strategies seem to be required and rating scales of a wide range of social behavior such as the MESSY could play an important role. Similarly, the selection of behaviors for treatment could be done in a more systematic fashion if methods such as normal rating scales and social-validation criteria are used to selected responses for treatment. Considerably more emphasis on research of this type by behavior therapists is warranted (Kazdin, Matson and Esveldt-Dawson, 1981).
REFERENCES Achenbach T. M. (1978) The Child Behavior Profile: I. Boys aged 6 through Il. J. cunstd~C&L Psychof,46, 478-488. Achenbach T. M. and Edelbrock C, S. (1979) The Child Behavior Profile: II. Boys aged 12-16 and girls aged 6-i I and 12-i& 2. C@tr.%&. &?I. Psj~&t?U~. 47, 223-233. Chittenden G. F. (1942) An experimenta study in measuring and rno~fyj~~ assertive behavior in young children. Moiqr. Sac. Rex Child Bev. 7, l-87. Combs M. L. and Slaby D. A. (1977) Social skills training with children. In Advances in Cfinieaf Chifd ~sycho~~~~, Vol. 1 (Edited by Lahey 3. B. and Kazdin A. E.). Plenum Press, New York. Connors C. A. (1969) A teacher rattng scale for use in drug studies with children. Am. J. Psychiar. 126, 152-156. Gersten J. C., Langner T. S., Eisenberg J. G., Simcha-Fagan 0. and McCartney E. D. (1976) Stability and change in types of behavioral disturbance of children and adolescents. J. abnorm. Child Psychol. 4, 1l-127. Hersen M. and Bellack A. S. (1977) Assessment of social skills. In Handbookfbr Behavior Assessmenr (Edited by Ceminero A. R., Calhoun K. S. and Adams H. E.). Wiley, New York. Kazdin A. E., Matson J. C. and &veldt-Dawson K. (1981) Social skill performance among normal and psychiatric inpatient children as a function of assessment conditions. B&au. Res. Ther. 19, 145-152. Keller M. F. and Carlson P. M. (1974) The use of symbolic modeling to promote social skills in preschool children with low levels of social responsiveness. Child Deu. 45, 912-919. Matson 3. L. and Senatore V. (t&l) A comp&ison of traditional psychotherapy and social sk& training fo; improving interpersonat functioning of mentally retarded adults. &&ru. Ther, 12, 369-382. Matson J. L., Esveldt-Dawson K., Andrasik F., Uhendick T. H., Petti T. and Hersen M. (1980) Direct, observational and generalization effects of social ski& training with emotionaily disturbed children. Behav. Ther. 11, 522-531. McFall R. M. (1982) A review and reformulation of the concept of sacial skills. Behav. Assess. 4, l-33. Morrison R. L. and Bellack A. S. (1981) The role of social perception in social skill. Behav. Ther. 12, 69-79. Quay H. C. (1977) Measuring dimensions of deviant behavior: the behavior problem checklist. J. abnorm. Child Psychol, 5, 177-189.
Quay H. C. and Peterson D. R. (1975) Manual for the behavior problem checklist. Unpublished manuscript. Reardon R. C.. Hersen M.. Be&k A. S. and Folev J. M. (1978) Measurinn social skill in made school bovs. d Unaublished L manuscript, University of Pittsburgh, Pennsylvania. . ’ Rinn R. C. and Markle A. (1980) Modification of skill deficits in children. In Re.rearch and Pracrice in Social Skills Training (Edited by Bellack A. S. and Hersen M.). Plenum Press, New York.
340
JOHNNY L. MATSON ef a/.
Schultz E. W., Salvia J. and Feinn J. (1973) Deviant behaviors in rural elementary school children. J. abnorm. Child Psychol. 4, 378-389. Stone F. B., Wilson M. A., Spence M. E. and Gibson R. C. (1969) A survey of elementary school children’s behavior problems. Am. J. Orthopsychiat. 39, 289-290. Van Hasselt V. B., Hersen M., Whitehall M. B. and Bellack A. S. (1979) Social skills assessment and training for children: an evaluative review. Behau. Res. Ther. 17, 413-437. Van Hasselt V. B., Hersen M. and Bellack A. S. (1981) The validity of role play tests for assessing social skills in children. Behav. Ther. 12, 2022216. Werry J. S. and Hawthorne D. (1976) Connor’s teachers questionnaire--norms and validity. Au&. N.Z. J. Psych&. 10, 257-262. Werry J. S. and Quay H. C. (1971) The prevalence of behavior symptoms in younger elementary school children. Am. J. Orih~p.~ych~al. 41. 136 143.