- Email: [email protected]
The analysis of reliability and validity of the IT-MAIS, MAIS and MUSS
Accepted Manuscript The Analysis of Reliability and Validity of the IT-MAIS, MAIS and MUSS Yan Zhong, Tianqiu Xu, Ruijuan Dong, Jing Lv, Bo Liu, Xueqing Chen PII:
S0165-5876(17)30097-6
DOI:
10.1016/j.ijporl.2017.03.006
Reference:
PEDOT 8450
To appear in:
International Journal of Pediatric Otorhinolaryngology
Received Date: 7 December 2016 Revised Date:
1 March 2017
Accepted Date: 2 March 2017
Please cite this article as: Y. Zhong, T. Xu, R. Dong, J. Lv, B. Liu, X. Chen, The Analysis of Reliability and Validity of the IT-MAIS, MAIS and MUSS, International Journal of Pediatric Otorhinolaryngology (2017), doi: 10.1016/j.ijporl.2017.03.006. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Title：The Analysis of Reliability and Validity of the IT-MAIS，MAIS and MUSS List of authors：
Tianqiu Xu
bachelor's degree
Ruijuan Dong Jing Lv
master's degree
Bo Liu
master's degree
Xueqing Chen
master's degree
AC C
EP
TE D
M AN U
bachelor's degree
Affiliation Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University
E-mail Address [email protected] om [email protected] m [email protected]
RI PT
Academic Degree bachelor's degree
[email protected]
SC
Name Yan Zhong
[email protected] xueqingchen2006 @aliyun.com
ACCEPTED MANUSCRIPT
The Analysis of Reliability and Validity of the IT-MAIS， ，MAIS and MUSS
RI PT
Yan Zhonga,b, Tianqiu Xua,b, Ruijuan Donga,b, Jing Lva,b, Bo Liua,b,1,** ,Xueqing Chena,b,1,* a
BeijingTongren Hospital, Capital Medical University, Beijing 100730, China Beijing Institute of Otolaryngology, Key Laboratory of Otolaryngology Head and Neck Surgery(Capital Medical University), Ministry of Education, Beijing, China *Corresponding author at: Beijing Tongren Hospital, Capital Medical University and Beijing Institute of Otolaryngology, 17 Hougou Lane, Chongnei Street, Dongcheng District, Beijing 100005, China Tel:+861058265805;Fax:+8610 85115988. E-mail address:[email protected]. **Corresponding author at: Beijing Tongren Hospital, Capital Medical University and Beijing Institute of Otolaryngology, 17 Hougou Lane, Chongnei Street, Dongcheng District, Beijing 100005, China Tel:+861058265816;Fax:+8610 85115988. E-mail address: [email protected]. 1 These authors contributed equally to this work.
M AN U
SC
b
TE D
Abstract
[Objective] The aim of this study was to investigate the reliability and validity of the Infant-toddler Meaningful Auditory Integration Scale
EP
(IT-MAIS), Meaningful Auditory Integration Scale (MAIS), and
AC C
Meaningful Use of Speech Scale (MUSS). [Methods] IT-MAIS, MAIS and MUSS were divided into 3 sub dimensions. 300 children with cochlear implants (CI) were included in the investigation. To assess test-retest reliability of these questionnaires, 30 children were selected randomly to be evaluated at a two-week interval indicated that there were no significant changes between test and retest. Furthermore random test analysis by different evaluators was also
ACCEPTED MANUSCRIPT
administered to 30 users. [Results] Reliability test: Test-retest reliability of the three scales was proved to be satisfactory. All domains had correlation coefficients that
RI PT
exceeded 0.750(P<0.01). The Cronbach’s α of the three scales and their three domains were greater than 0.700. Reliability between evaluators of the three scales were considered to be satisfactory. All domains had
SC
correlation coefficients that exceeded 0.750(P<0.01). Validity test: The
M AN U
evaluation of content validity by expert review showed the questionnaire had good content validity; The correlation coefficients between the overall scores of the three scales and their three domains were 0.699~0.978(P<0.01). There were correlations among the three
TE D
sub-domains but the strength of the correlations was relatively low. There was certain construct validity.
[Conclusions] IT-MAIS, MAIS, MUSS scales have good reliability and
EP
validity, and can be used to measure the outcome for children with
AC C
cochlear implants hearing and speech evaluation. [Key words] Cochlear implantation；Questionnaires； Auditory ability； Speech ability；Reliability；Validity.
Introduction As the growing popularity of newborn hearing screening and diagnostic hearing technology continues to improve and various assistive
ACCEPTED MANUSCRIPT
listening devices such as hearing aids (HA) and cochlear implants (CI) continue to improve, more and more deaf children can be diagnosed accurately in a relatively short period of time after birth. Early detection,
RI PT
diagnosis and intervention are now becoming possible. This surely would benefit deaf children. However as the implant ages are becoming increasingly younger, it can be difficult using conventional rehabilitation
SC
assessments such as pure tone audiometry, speech recognition, etc due to
M AN U
inability to cooperate among the young patients. This creates a need for a series of measures that can be used to assess hearing and speech rehabilitation without requiring their cooperation [1,2]. Several questionnaires are utilized and the most commonly used are the IT-MAIS,
TE D
MAIS and MUSS. MAIS was completed in 1991 by Robbins et al. [3] at the Indiana School of Medicine in the United States. In 1997 Osberger et al. [4] revised MAIS and proposed IT-MAIS scale based on the
EP
characteristics of infants. IT-MAIS scale is mainly used to assess infants
AC C
hearing ability. In 1992 Robbins et al. put forward MUSS scale to assess children's verbal output capacity in everyday life. These questionnaires have been translated into several languages and used in many countries. [5] The questionnaires need to be evaluated to see if they have the ability to achieve the intended objectives through measuring its reliability and validity. We hope that they have the capability of reflecting a more realistic assessment of hearing and speech abilities for children in their
ACCEPTED MANUSCRIPT
natural social environment. Their reliability and validity have been investigated with satisfactory results in English, German, and Polish versions [5]. The investigation of reliability and validity of the scales in
RI PT
Chinese version is necessary to better evaluate the hearing and speech capabilities in children before and after a hearing aid or cochlear implant. This can also be used to guide the auditory rehabilitation. Yun Zheng et al.
SC
concluded that the Chinese version of IT-MAIS and MAIS had good
M AN U
reliability and validity when they were used in hearing and speech assessment of normal children [6]. This study was conducted to assess the reliability and validity of cochlear implants for children since these scales are quite commonly used in clinical settings to assess hearing and speech
TE D
of children with CI.
Subjects
EP
Material and methods
AC C
We conducted the study at the CI center in the Tongren Hospital. The study subjects were children implanted CI before the age of 8 years. All children were born with profound sensorineural hearing loss (SNHL). All caregivers reported that Chinese were the primary language used at home. 317 children were evaluated, and 300 (146 boys and 154 girls) of them met the criteria. 17 children were excluded from this study(refer to the measurements for detailed information). 100 children (45 boys and 55
ACCEPTED MANUSCRIPT
girls) who were less than 8 years old and before 3 years by the times of the cochlear implant were evaluated by using IT-MAIS. 100 children (58 boys and 42 girls) who implanted CI at the age of 3-8 years and evaluated
RI PT
before age 8 by using MAIS. The last 100 study subjects (43 boys and 57 girls) who implanted CI and evaluated before age 8 by the use of MUSS.
SC
Test materials
M AN U
(1) The IT-MAIS is used to evaluate the listening skills in infants [7]. It includes 10 items grouped into 3 domains: Vocalization Behavior (item1~2), Alerting to Sounds (item3~6), and Deriving Meaning from Sound (item7~10) [8]. Each item is divided into five levels and scored
TE D
from 0 to 4: 0 = never, 1 = rarely (25%), 2 = occasionally (50%), 3 = frequently (75%), and 4 = always (100%) [9]. (2)The MAIS is used to assess listening skills in children. It also includes
to
Sounds
(item3~6),
and
Deriving
Meaning
from
AC C
Alerting
EP
10 items grouped into 3 domains: Self-report about the Device (item1~2),
Sound(item7~10). The scoring formula is the same as the IT-MAIS. (3)The MUSS is used to assess children’s verbal ability in natural social environment. The 10 items of the scale are divided into 3 domains: the sound generation (item1~3), the communication capability (item4~7), and the communication skills (item8~10). Each item is divided into five levels and scored from 0 to 4: 0 = never, 1 = rarely (<50%), 2 =
ACCEPTED MANUSCRIPT
occasionally (>50%), 3 = frequently (>75%), and 4 = always (100%). Measurements The evaluators were trained audiology professionals who received
RI PT
training on how to administer the scales before the scales’ administration. Before the interview caregivers were taught that a few questions about their child’s spontaneous responses to sounds in their natural environment
SC
would be inquired. Each of the 10 questions in the scale was asked one by
M AN U
one by the evaluators and caregivers of the children made detailed descriptions of the children’s spontaneous auditory and linguistic reactions in their daily lives. The surveys were conducted at regular intervals by audiologists face to face or telephone with caregivers. The
TE D
evaluators recorded the parents’ descriptions of their children and derived scores according to the recorded descriptions [9]. The first evaluation：100 children were evaluated and the scores of
AC C
total score.
EP
each item were calculated including score of the three domains and the
The re-evaluation: after 2 weeks, 30 children were randomly
selected and evaluated again and the scores were calculated. The test-retest reliability coefficients of the three domains and the total scale were calculated as well. Evaluation of different personnel: again 30 children were randomly selected and evaluated by another audiology professional who
ACCEPTED MANUSCRIPT
was familiar with these questionnaires and the scores were obtained. The coefficient between the two evaluators was calculated. Evaluators determined the score for each item answered by
RI PT
caregivers. A "no answer" was marked if the parent or teacher could not answer the question. The test was considered invalid if a caregiver could not answer the question more than twice.
SC
Statistics
M AN U
SPSS 13.0 software was used for data analysis. A P-value <0.05 was considered statistically significant. Reliability
(1) test-retest reliability: using intra-class correlation coefficient (ICC)
TE D
test, the coefficient greater than 0.750 was considered acceptable. (2) internal consistency: Cronbach's α coefficient was used to assess the internal consistency of each domains and the total scale. The higher the
EP
coefficient, the better the internal consistency of the scale. Cronbach's
AC C
α coefficient greater than 0.700 is acceptable. (3) reliability between evaluators: ICC was used in the test. The coefficient greater than 0.750 was considered acceptable. Validity
(1) content validity was evaluated by the use of expert judgment method logical analysis which uses the expert scoring method by deleting the entries with low content validity.
ACCEPTED MANUSCRIPT
(2) construct validity was assessed by using the internal consistency test. There should be a moderate to high correlation between each of the domains and the overall scale and the correlation among different
RI PT
domains is relatively low. Results Reliability
SC
(1)The test-retest reliability
M AN U
The test-retest reliability of the 3 scales showed high ICC values ranged from 0.882 to 0.929. All the scales’ total score test-retest reliability ICCs and there three domains’ ICCs are greater than 0.750, as showing in Table 1-1, Table 1-2 and Table 1-3, they had a good test-retest reliability.
Value
0.929
AC C
ICC value
Overall
EP
Statistics
TE D
Table 1-1: IT-MAIS Test-retest Reliability Coefficients (n=30)
P value
<0.01
Vocalization
Alerting to
Deriving
Behavior
Sounds
Meaning from Sounds
0.902
0.832
0.875
<0.01
<0.01
<0.01
Table 1-2: MAIS Test-retest Reliability Coefficients (n=30) Statistics Value
Overall
Self-report
Alerting to
Deriving
about the
Sounds
Meaning
Device
from Sounds
ACCEPTED MANUSCRIPT
ICC value
0.921
0.903
0.854
0.890
P value
<0.01
<0.01
<0.01
<0.01
Overall
Value
Sound
Communication Communication
Generation
Capability
0.928
0.893
0.900
P value
<0.01
<0.01
<0.01
0.898
<0.01
M AN U
ICC value
Skills
SC
Statistics
RI PT
Table 1-3 MUSS Test-retest Reliability Coefficients (n=30)
(2) Internal consistency analysis
The result showed that the total score of the scale’s Cronbach's α
TE D
coefficients were greater than 0.700 and the internal consistency in three domains was adequate. The results were shown in Table 2-1, Table 2-2,
EP
and Table 2-3.
Table 2-1 IT-MAIS Cronbach’s α Coefficients（n=100）
AC C
Statistics
Overall Vocalization Behavior Alerting to Sounds Deriving Meaning from Sounds
Average
Cronbach’s α
26.84 6.84
Standard Deviation 11.281 1.868
11.58
4.780
0.762
8.42
5.474
0.747
0.836 0.888
ACCEPTED MANUSCRIPT
Table 2-2 MAIS Cronbach’s α Coefficients（n=100）
Overall Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
28.07 6.99
Standard Deviation 8.757 1.418
12.03
3.600
9.05
4.850
Cronbach’s α 0.747 0.881
RI PT
Average
0.754
0.702
M AN U
SC
Statistics
Table 2-3 MUSS Cronbach’s α Coefficients（n=100） Average
Overall Sound Generation
18.85 8.79
Communication
6.97
EP
Capability
TE D
Statistics
3.09
Cronbach’s α
5.555
0.743
3.009
0.856
0.886 0.839
AC C
Communication Skills
Standard Deviation 11.200 3.382
（3）Reliability between evaluators The results showed high values between 0.74 and 0.80. The MAIS showed the best reliability, with an ICC of 0.801. The ICC of IT-MAIS was 0.749 and of MUSS 0.997. We can see that all the scales’ total score reliability between evaluators ICC value and its three domains are greater
ACCEPTED MANUSCRIPT
than 0.750, as they are shown in Table 3-1, Table 3-2 and Table 3-3. Table 3-1 IT-MAIS reliability between evaluators（n=30） Overall
Vocalization Behavior
Alerting to Sounds
ICC value
0.894
0.813
0.870
Deriving Meaning from Sounds 0.842
P value
<0.01
<0.01
<0.01
<0.01
RI PT
Statistics
SC
Table 3-2
Table 3-2: MAIS reliability between evaluators (n=30) Overall
Self-report
Alerting to
Deriving
about the
Sounds
Meaning
M AN U
Statistics Value
Device 0.889
P value
<0.01
0.811
TE D
ICC value
<0.01
from Sounds
0.840
0.825
<0.01
<0.01
Overall
AC C
Statistics
EP
Table 3-3 MUSS reliability between evaluators (n=30)
Value
Sound
Communication Communication
Generation
Capability
Skills
ICC value
0.910
0.834
0.876
0.898
P value
<0.01
<0.01
<0.01
<0.01
Validity (1) Content validity Four experts believed that the content of the items of the three scales
ACCEPTED MANUSCRIPT
were in line with the measurement requirements, the representativeness of each item, and the content validity were acceptable. This means that the content validity of the 3 scales was good.
RI PT
(2) Construct validity Table 4-1 IT-MAIS Pearson Correlation Analysis（n=100） Vocalization Behavior
Vocalization Behavior Alerting to Sounds Deriving Meaning from Sounds
0.841
--
0.952
--
Deriving Meaning from Sounds --
0.809
--
--
0.685
0.812
--
TE D
0.943
Alerting to Sounds
SC
Overall
M AN U
Statistics
Table 4-2 MAIS Pearson Correlation Analysis（n=100） Overall
Alerting to Sounds
0.699
Self-report about the Device --
--
Deriving Meaning from Sounds --
0.904
0.612
--
--
0.930
0.516
0.712
--
EP
Statistics
AC C
Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
ACCEPTED MANUSCRIPT
Table 4-3 MUSS Pearson Correlation Analysis（n=100） Over all
Sound Generation
Sound
Communication
Communication
Generation
Capability
Skills
--
--
--
0.851
--
0.693
0.838
0.910
Communication 0.978
--
--
M AN U
Communication 0.893 Skills
SC
Capability
RI PT
Statistics
Discussion
Children are undergoing cochlear implantation at younger and
TE D
younger ages. This decline in age and the challenges associated with accurately assessing the functional hearing of infants raises concerns regarding the tools used for CI candidacy evaluation and post-CI progress
EP
assessments. The three scales, IT-MAIS, MAIS and MUSS, have been
AC C
very widely used by audiologists and speech-language pathologists in our clinical work, they can provide benefits for deaf children, including diagnosis, setting treatment goals, and tracking progress. As caregiver report tools, parents can also measure listening and speaking skills in children at home fast and conveniently. However, they have not undergone rigorous psychometric analysis, this present study was undertaken to investigate the reliability and validity of these three scales.
ACCEPTED MANUSCRIPT
Reliability Reliability is the consistency or stability of a measure. A measure is
RI PT
considered to have a high reliability if it produces similar results under consistent conditions. Reliability is expressed by correlation coefficient which can be divided into the stability coefficient and internal
SC
consistency coefficient. [12]
M AN U
Reliability in stability coefficient consists of test-retest reliability and reliability between evaluators, measured by intraclass correlation coefficient (ICC). The ICC was first used in 1966 by Bartko for the measurement and evaluation of reliability of size. The value below 0.4
TE D
indicates poor reliability, greater than 0.75 indicate a good reliability. [10] Test-retest reliability was used to evaluate the same questionnaire for the same group of children to a certain time interval repeat evaluation.
EP
The three scales’ overall measure showed excellent test-retest reliability
AC C
(0.921-0.929) and the three domains of each scale showed moderate reliability (0.832-0.903). The results showed that all the three scales’ test-retest reliability correlation coefficient of the whole scale and 3 domains are greater than 0.750, the three scales have acceptable test-retest reliability. Reliability between evaluators evaluated the same subjects by different evaluators without discussion. In this assessment, the ICC value of the overall scale of IT-MAIS、MAIS and MUSS are
ACCEPTED MANUSCRIPT
0.894, 0.889 and 0.910, greater than 0.750.While, the three domains of the three scales are all acceptable. The three scales have good reliability between evaluators.
RI PT
Reliability in internal consistency is assessed by Cronbach’s α. It reflects a measure of the relationship among all the items. High Cronbach’s α suggests that items have a close relationship and they are all
SC
measuring different aspects of the same construct. An acceptable
M AN U
reliability should greater than 0.7 or above. The results shows that the three scales have an excellent internal consistency. Validity
Validity refers to the degree of correctness, that is, a scale can
TE D
accurately measure what it is intended to measure which include effectiveness and correctness of the scale. Validity is an important condition for a scientific test tool to be useful.
EP
Content validity is a subjective indicator which refers to the
AC C
measurement of content and compliance for the selected topic, measurement is consistent with the measurement objective and requirement. In this study, three scales of each item evaluation by experts, considering all the items are in compliance with the measurement objective and requirement, have good content validity. Construct validity refers to the questionnaire can measure the characteristics of the theoretically expected degree. Evaluation methods
ACCEPTED MANUSCRIPT
are factor analysis, internal consistency test etc. This study selected the internal consistency to test construct validity of the scale. It meets the requirements of psychometrics if the correlation coefficient of each
RI PT
domain was 0.3~0.5 and the correlation coefficient between each domain and the whole scale was 0.5~0.8. The analysis showed that the IT-MAIS scale 3 domains with the total scale of correlation were between
SC
0.841~0.952 and relationship between the 3 domains were among
M AN U
0.685~0.812, 3 domains with the total scale correlation is very good. There was some correlation among the 3 domains but the correlation was less that of the total scale. The same results occured in the other two scales.
TE D
As we can see in the results, some domains in the three scales correlation coefficients are much higher, this may be because children's auditory and speech development is a continuous process, there is a close between
EP
relationship
listening
development
and
language
AC C
development.[13]scales have certain continuity between each sub dimensions. For IT-MAIS and MAIS, the correlation coefficient of alerting to sounds and deriving meaning from sound is high. This is probably because the auditory development in infants is step by step, well listening ability are more likely to exhibit better understanding ability, good ability of understanding based on a better awareness, it has relevance itself, and may lead to high correlation coefficient between
ACCEPTED MANUSCRIPT
different domains. The MAIS, the self-report about the device has relatively low correlations with the whole scale and the other two domains, our consideration is that how they use their devices is children's
RI PT
habits, different children on device preferences and habits are different, so the relationship is relatively low, but still within the normal range. The MUSS measures infants and children's verbal ability. Its three domains
SC
are voice, communication capability and communication skills. It has a
M AN U
very strong continuity because children often first learn to say something, and then to communicate and use some skills. The MUSS also has strong correlation. In conclusion, the three scales have acceptable construct validity.
TE D
The results provide strong evidence that the three scales possess good reliability and appropriate validity, and can be used to measure the outcome of cochlear implant in children. Considering the limited sample
EP
size, we can expand the sample size for further discussion and research.
AC C
Acknowledgements
Thanks to all the children and their families who participated in the
study. We would like to gratefully acknowledge Wenfang Wu from Capital Medical University for the statistical analyses. The project was supported by the capital citizen health program to foster from Beijing Municipal Science & Technology Commission (No.Z141100002114033), the Beijing Municipal Administration of Hospitals Clinical medicine
ACCEPTED MANUSCRIPT
Development of special funding support (No.XMLX201514), the National Natural Science Foundation of China (No.81371103), the capital health research and development of special from the Beijing Municipal
RI PT
Health Bureau (No.2011-1017-01). Thanks for the hard work on the training center of Beijing artificial cochlear language and the training
SC
center of China and Australia. References
years:
M AN U
[1] Osberger MJ. Cochlear implantation in children under the age of two candidacy
considerations.
1997;117:145–9.
Otolaryngol
Head
Neck
Surg
[2] Valencia DM, Rimell FL, Friedman BJ, Oblander MR, Helmbrecht J.
TE D
Cochlear implantation in infants less than 12 months of age. Int J Pediatr Otorhinolaryngol 2008;72:767–73.
EP
[3]Robbins A M, Rnesh A W J J, Berry S W. Evaluating meaningful auditory integration in profoundly hearing impaired children [J].Am J
AC C
Otol,1991,12 Suppl:144~150. [4]Osberger M J, Geier L, Zimmerman Phillips S, et al. Use of a parent-report scale to assess benefit in children given the Clarion cochlear implant [J].Am J Otol,1997,18(6 Suppl):S79~80. [5]Weichbold V, Anderson H, Haese PD: Validation of three adaptations of the Meaningful Auditory Integration Scale (MAIS) to German, English and Polish. Int J Audiol 2004; 43: 156–161.
ACCEPTED MANUSCRIPT
[6] Yun Zheng, Sigfrid D. Soli, Kai Wang, et al. A normative study of early prelingual auditory development. Audiol Neurotol 2009;14:214– 222.
RI PT
[7] Anne Schubert.Examining the validity and reliability of the IT-MAIS via RASCH analysis. B.A.University of Michigan, 2013.
[8]Zimmerman-Phillips, S., Osberger, M. J., & Robbins, A. M. (2001).
M AN U
Advanced Bionics Corporation.
SC
Infant-Toddler Meaningful Auditory Integration Scale. Sylmar, CA:
[9]Xueqing Chen, Sha Liu, Bo Liu1, et al. The effects of age at cochlear implantation and hearing aid trial on auditory performance of Chinese infants. Acta Oto-Laryngologica, 2010; 130: 263–270.
TE D
[10]Bartko JJ. The intraclass co rrelation coefficient as a measure of reliability.P sycho l Rep, 1966; 19:3.
[11]Revelle. W., Zinbarg, R.E. Coefficients alpha, beta, omega, and the
EP
glb: Comments on Sijtsma.Psychometrika,2009,74:145~154.
AC C
[12]Hinderink JB, Krabbe PF, Van Den Broek P Development and application of a health-related quality-oflife instrument for adults with cochlear implants: the Nijmegen cochlear implant questionnaire,2000. [13]Arlinger, S., Lunner, T., Lyxell, B., & Pichora-Fuller, K., M. (2009). The emergence of cognitive hearing science. Scandinavian Journal of Psychology, 50, 371-384. doi: 10.1111/j.1467- 9450.2009.00753.
ACCEPTED MANUSCRIPT
Table 1-1: IT-MAIS Test-retest Reliability Coefficients (n=30) Statistics
Overall
Value
Vocalization
Alerting to
Deriving
Behavior
Sounds
Meaning
0.929
0.902
0.832
P value
<0.01
<0.01
<0.01
0.875 <0.01
SC
ICC value
RI PT
from Sounds
Statistics
M AN U
Table 1-2: MAIS Test-retest Reliability Coefficients (n=30) Overall
Value
Self-report
Alerting to
Deriving
about the
Sounds
Meaning
Device 0.921
P value
<0.01
0.903
TE D
ICC value
EP
<0.01
from Sounds
0.854
0.890
<0.01
<0.01
Table 1-3 MUSS Test-retest Reliability Coefficients (n=30) Overall
AC C
Statistics Value
Sound
Communication Communication
Generation
Capability
Skills
ICC value
0.928
0.893
0.900
0.898
P value
<0.01
<0.01
<0.01
<0.01
Table 2-1 IT-MAIS Cronbach’s α Coefficients（n=100） Statistics
Average
Overall
26.84
Standard Deviation 11.281
Cronbach’s α 0.836
ACCEPTED MANUSCRIPT
6.84
1.868
0.888
11.58
4.780
0.762
8.42
5.474
0.747
RI PT
Vocalization Behavior Alerting to Sounds Deriving Meaning from Sounds
Table 2-2 MAIS Cronbach’s α Coefficients（n=100） Average
Overall Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
28.07 6.99
Cronbach’s α 0.747 0.881
3.600
0.754
4.850
0.702
TE D
9.05
M AN U
12.03
Standard Deviation 8.757 1.418
SC
Statistics
EP
Table 2-3 MUSS Cronbach’s α Coefficients（n=100）
Overall Sound Generation
18.85 8.79
Standard Deviation 11.200 3.382
Communication
6.97
5.555
0.743
3.09
3.009
0.856
AC C
Statistics
Average
Cronbach’s α 0.886 0.839
Capability Communication Skills
Table 3-1 IT-MAIS reliability between evaluators（n=30）
ACCEPTED MANUSCRIPT
Overall
Vocalization Behavior
Alerting to Sounds
0.894
0.813
0.870
Deriving Meaning from Sounds 0.842
<0.01
<0.01
<0.01
<0.01
ICC value P value
RI PT
Statistics
Table 3-2
Statistics
Overall
Value
Self-report
Alerting to
Deriving
about the
SC
Table 3-2: MAIS reliability between evaluators (n=30)
Sounds
Meaning
M AN U
Device
from Sounds
0.889
0.811
0.840
0.825
P value
<0.01
<0.01
<0.01
<0.01
TE D
ICC value
Table 3-3 MUSS reliability between evaluators (n=30)
Value
0.910
AC C
ICC value
Overall
Sound
Communication Communication
Generation
Capability
Skills
0.834
0.876
0.898
<0.01
<0.01
<0.01
EP
Statistics
P value
<0.01
Table 4-1 IT-MAIS Pearson Correlation Analysis（n=100）
Statistics
Overall
Vocalization Behavior
Alerting to Sounds
Vocalization Behavior Alerting to
0.841
--
--
Deriving Meaning from Sounds --
0.952
0.809
--
--
ACCEPTED MANUSCRIPT
0.943
0.685
0.812
--
RI PT
Sounds Deriving Meaning from Sounds
Table 4-2 MAIS Pearson Correlation Analysis（n=100）
Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
0.699
Self-report about the Device --
Alerting to Sounds
SC
Overall
--
Deriving Meaning from Sounds --
M AN U
Statistics
0.904
--
--
0.516
0.712
--
TE D
0.930
0.612
Table 4-3 MUSS Pearson Correlation Analysis（n=100） Overa ll
Sound
Communication
Communication
Generation
Capability
Skills
--
--
--
0.851
--
--
0.693
0.838
--
AC C
EP
Statistics
Sound Generation
0.910
Communication 0.978 Capability Communication 0.893 Skills
S0165-5876(17)30097-6
DOI:
10.1016/j.ijporl.2017.03.006
Reference:
PEDOT 8450
To appear in:
International Journal of Pediatric Otorhinolaryngology
Received Date: 7 December 2016 Revised Date:
1 March 2017
Accepted Date: 2 March 2017
Please cite this article as: Y. Zhong, T. Xu, R. Dong, J. Lv, B. Liu, X. Chen, The Analysis of Reliability and Validity of the IT-MAIS, MAIS and MUSS, International Journal of Pediatric Otorhinolaryngology (2017), doi: 10.1016/j.ijporl.2017.03.006. This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Title：The Analysis of Reliability and Validity of the IT-MAIS，MAIS and MUSS List of authors：
Tianqiu Xu
bachelor's degree
Ruijuan Dong Jing Lv
master's degree
Bo Liu
master's degree
Xueqing Chen
master's degree
AC C
EP
TE D
M AN U
bachelor's degree
Affiliation Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University Beijing Tongren Hospital Capital Medical University
E-mail Address [email protected] om [email protected] m [email protected]
RI PT
Academic Degree bachelor's degree
[email protected]
SC
Name Yan Zhong
[email protected] xueqingchen2006 @aliyun.com
ACCEPTED MANUSCRIPT
The Analysis of Reliability and Validity of the IT-MAIS， ，MAIS and MUSS
RI PT
Yan Zhonga,b, Tianqiu Xua,b, Ruijuan Donga,b, Jing Lva,b, Bo Liua,b,1,** ,Xueqing Chena,b,1,* a
BeijingTongren Hospital, Capital Medical University, Beijing 100730, China Beijing Institute of Otolaryngology, Key Laboratory of Otolaryngology Head and Neck Surgery(Capital Medical University), Ministry of Education, Beijing, China *Corresponding author at: Beijing Tongren Hospital, Capital Medical University and Beijing Institute of Otolaryngology, 17 Hougou Lane, Chongnei Street, Dongcheng District, Beijing 100005, China Tel:+861058265805;Fax:+8610 85115988. E-mail address:[email protected]. **Corresponding author at: Beijing Tongren Hospital, Capital Medical University and Beijing Institute of Otolaryngology, 17 Hougou Lane, Chongnei Street, Dongcheng District, Beijing 100005, China Tel:+861058265816;Fax:+8610 85115988. E-mail address: [email protected]. 1 These authors contributed equally to this work.
M AN U
SC
b
TE D
Abstract
[Objective] The aim of this study was to investigate the reliability and validity of the Infant-toddler Meaningful Auditory Integration Scale
EP
(IT-MAIS), Meaningful Auditory Integration Scale (MAIS), and
AC C
Meaningful Use of Speech Scale (MUSS). [Methods] IT-MAIS, MAIS and MUSS were divided into 3 sub dimensions. 300 children with cochlear implants (CI) were included in the investigation. To assess test-retest reliability of these questionnaires, 30 children were selected randomly to be evaluated at a two-week interval indicated that there were no significant changes between test and retest. Furthermore random test analysis by different evaluators was also
ACCEPTED MANUSCRIPT
administered to 30 users. [Results] Reliability test: Test-retest reliability of the three scales was proved to be satisfactory. All domains had correlation coefficients that
RI PT
exceeded 0.750(P<0.01). The Cronbach’s α of the three scales and their three domains were greater than 0.700. Reliability between evaluators of the three scales were considered to be satisfactory. All domains had
SC
correlation coefficients that exceeded 0.750(P<0.01). Validity test: The
M AN U
evaluation of content validity by expert review showed the questionnaire had good content validity; The correlation coefficients between the overall scores of the three scales and their three domains were 0.699~0.978(P<0.01). There were correlations among the three
TE D
sub-domains but the strength of the correlations was relatively low. There was certain construct validity.
[Conclusions] IT-MAIS, MAIS, MUSS scales have good reliability and
EP
validity, and can be used to measure the outcome for children with
AC C
cochlear implants hearing and speech evaluation. [Key words] Cochlear implantation；Questionnaires； Auditory ability； Speech ability；Reliability；Validity.
Introduction As the growing popularity of newborn hearing screening and diagnostic hearing technology continues to improve and various assistive
ACCEPTED MANUSCRIPT
listening devices such as hearing aids (HA) and cochlear implants (CI) continue to improve, more and more deaf children can be diagnosed accurately in a relatively short period of time after birth. Early detection,
RI PT
diagnosis and intervention are now becoming possible. This surely would benefit deaf children. However as the implant ages are becoming increasingly younger, it can be difficult using conventional rehabilitation
SC
assessments such as pure tone audiometry, speech recognition, etc due to
M AN U
inability to cooperate among the young patients. This creates a need for a series of measures that can be used to assess hearing and speech rehabilitation without requiring their cooperation [1,2]. Several questionnaires are utilized and the most commonly used are the IT-MAIS,
TE D
MAIS and MUSS. MAIS was completed in 1991 by Robbins et al. [3] at the Indiana School of Medicine in the United States. In 1997 Osberger et al. [4] revised MAIS and proposed IT-MAIS scale based on the
EP
characteristics of infants. IT-MAIS scale is mainly used to assess infants
AC C
hearing ability. In 1992 Robbins et al. put forward MUSS scale to assess children's verbal output capacity in everyday life. These questionnaires have been translated into several languages and used in many countries. [5] The questionnaires need to be evaluated to see if they have the ability to achieve the intended objectives through measuring its reliability and validity. We hope that they have the capability of reflecting a more realistic assessment of hearing and speech abilities for children in their
ACCEPTED MANUSCRIPT
natural social environment. Their reliability and validity have been investigated with satisfactory results in English, German, and Polish versions [5]. The investigation of reliability and validity of the scales in
RI PT
Chinese version is necessary to better evaluate the hearing and speech capabilities in children before and after a hearing aid or cochlear implant. This can also be used to guide the auditory rehabilitation. Yun Zheng et al.
SC
concluded that the Chinese version of IT-MAIS and MAIS had good
M AN U
reliability and validity when they were used in hearing and speech assessment of normal children [6]. This study was conducted to assess the reliability and validity of cochlear implants for children since these scales are quite commonly used in clinical settings to assess hearing and speech
TE D
of children with CI.
Subjects
EP
Material and methods
AC C
We conducted the study at the CI center in the Tongren Hospital. The study subjects were children implanted CI before the age of 8 years. All children were born with profound sensorineural hearing loss (SNHL). All caregivers reported that Chinese were the primary language used at home. 317 children were evaluated, and 300 (146 boys and 154 girls) of them met the criteria. 17 children were excluded from this study(refer to the measurements for detailed information). 100 children (45 boys and 55
ACCEPTED MANUSCRIPT
girls) who were less than 8 years old and before 3 years by the times of the cochlear implant were evaluated by using IT-MAIS. 100 children (58 boys and 42 girls) who implanted CI at the age of 3-8 years and evaluated
RI PT
before age 8 by using MAIS. The last 100 study subjects (43 boys and 57 girls) who implanted CI and evaluated before age 8 by the use of MUSS.
SC
Test materials
M AN U
(1) The IT-MAIS is used to evaluate the listening skills in infants [7]. It includes 10 items grouped into 3 domains: Vocalization Behavior (item1~2), Alerting to Sounds (item3~6), and Deriving Meaning from Sound (item7~10) [8]. Each item is divided into five levels and scored
TE D
from 0 to 4: 0 = never, 1 = rarely (25%), 2 = occasionally (50%), 3 = frequently (75%), and 4 = always (100%) [9]. (2)The MAIS is used to assess listening skills in children. It also includes
to
Sounds
(item3~6),
and
Deriving
Meaning
from
AC C
Alerting
EP
10 items grouped into 3 domains: Self-report about the Device (item1~2),
Sound(item7~10). The scoring formula is the same as the IT-MAIS. (3)The MUSS is used to assess children’s verbal ability in natural social environment. The 10 items of the scale are divided into 3 domains: the sound generation (item1~3), the communication capability (item4~7), and the communication skills (item8~10). Each item is divided into five levels and scored from 0 to 4: 0 = never, 1 = rarely (<50%), 2 =
ACCEPTED MANUSCRIPT
occasionally (>50%), 3 = frequently (>75%), and 4 = always (100%). Measurements The evaluators were trained audiology professionals who received
RI PT
training on how to administer the scales before the scales’ administration. Before the interview caregivers were taught that a few questions about their child’s spontaneous responses to sounds in their natural environment
SC
would be inquired. Each of the 10 questions in the scale was asked one by
M AN U
one by the evaluators and caregivers of the children made detailed descriptions of the children’s spontaneous auditory and linguistic reactions in their daily lives. The surveys were conducted at regular intervals by audiologists face to face or telephone with caregivers. The
TE D
evaluators recorded the parents’ descriptions of their children and derived scores according to the recorded descriptions [9]. The first evaluation：100 children were evaluated and the scores of
AC C
total score.
EP
each item were calculated including score of the three domains and the
The re-evaluation: after 2 weeks, 30 children were randomly
selected and evaluated again and the scores were calculated. The test-retest reliability coefficients of the three domains and the total scale were calculated as well. Evaluation of different personnel: again 30 children were randomly selected and evaluated by another audiology professional who
ACCEPTED MANUSCRIPT
was familiar with these questionnaires and the scores were obtained. The coefficient between the two evaluators was calculated. Evaluators determined the score for each item answered by
RI PT
caregivers. A "no answer" was marked if the parent or teacher could not answer the question. The test was considered invalid if a caregiver could not answer the question more than twice.
SC
Statistics
M AN U
SPSS 13.0 software was used for data analysis. A P-value <0.05 was considered statistically significant. Reliability
(1) test-retest reliability: using intra-class correlation coefficient (ICC)
TE D
test, the coefficient greater than 0.750 was considered acceptable. (2) internal consistency: Cronbach's α coefficient was used to assess the internal consistency of each domains and the total scale. The higher the
EP
coefficient, the better the internal consistency of the scale. Cronbach's
AC C
α coefficient greater than 0.700 is acceptable. (3) reliability between evaluators: ICC was used in the test. The coefficient greater than 0.750 was considered acceptable. Validity
(1) content validity was evaluated by the use of expert judgment method logical analysis which uses the expert scoring method by deleting the entries with low content validity.
ACCEPTED MANUSCRIPT
(2) construct validity was assessed by using the internal consistency test. There should be a moderate to high correlation between each of the domains and the overall scale and the correlation among different
RI PT
domains is relatively low. Results Reliability
SC
(1)The test-retest reliability
M AN U
The test-retest reliability of the 3 scales showed high ICC values ranged from 0.882 to 0.929. All the scales’ total score test-retest reliability ICCs and there three domains’ ICCs are greater than 0.750, as showing in Table 1-1, Table 1-2 and Table 1-3, they had a good test-retest reliability.
Value
0.929
AC C
ICC value
Overall
EP
Statistics
TE D
Table 1-1: IT-MAIS Test-retest Reliability Coefficients (n=30)
P value
<0.01
Vocalization
Alerting to
Deriving
Behavior
Sounds
Meaning from Sounds
0.902
0.832
0.875
<0.01
<0.01
<0.01
Table 1-2: MAIS Test-retest Reliability Coefficients (n=30) Statistics Value
Overall
Self-report
Alerting to
Deriving
about the
Sounds
Meaning
Device
from Sounds
ACCEPTED MANUSCRIPT
ICC value
0.921
0.903
0.854
0.890
P value
<0.01
<0.01
<0.01
<0.01
Overall
Value
Sound
Communication Communication
Generation
Capability
0.928
0.893
0.900
P value
<0.01
<0.01
<0.01
0.898
<0.01
M AN U
ICC value
Skills
SC
Statistics
RI PT
Table 1-3 MUSS Test-retest Reliability Coefficients (n=30)
(2) Internal consistency analysis
The result showed that the total score of the scale’s Cronbach's α
TE D
coefficients were greater than 0.700 and the internal consistency in three domains was adequate. The results were shown in Table 2-1, Table 2-2,
EP
and Table 2-3.
Table 2-1 IT-MAIS Cronbach’s α Coefficients（n=100）
AC C
Statistics
Overall Vocalization Behavior Alerting to Sounds Deriving Meaning from Sounds
Average
Cronbach’s α
26.84 6.84
Standard Deviation 11.281 1.868
11.58
4.780
0.762
8.42
5.474
0.747
0.836 0.888
ACCEPTED MANUSCRIPT
Table 2-2 MAIS Cronbach’s α Coefficients（n=100）
Overall Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
28.07 6.99
Standard Deviation 8.757 1.418
12.03
3.600
9.05
4.850
Cronbach’s α 0.747 0.881
RI PT
Average
0.754
0.702
M AN U
SC
Statistics
Table 2-3 MUSS Cronbach’s α Coefficients（n=100） Average
Overall Sound Generation
18.85 8.79
Communication
6.97
EP
Capability
TE D
Statistics
3.09
Cronbach’s α
5.555
0.743
3.009
0.856
0.886 0.839
AC C
Communication Skills
Standard Deviation 11.200 3.382
（3）Reliability between evaluators The results showed high values between 0.74 and 0.80. The MAIS showed the best reliability, with an ICC of 0.801. The ICC of IT-MAIS was 0.749 and of MUSS 0.997. We can see that all the scales’ total score reliability between evaluators ICC value and its three domains are greater
ACCEPTED MANUSCRIPT
than 0.750, as they are shown in Table 3-1, Table 3-2 and Table 3-3. Table 3-1 IT-MAIS reliability between evaluators（n=30） Overall
Vocalization Behavior
Alerting to Sounds
ICC value
0.894
0.813
0.870
Deriving Meaning from Sounds 0.842
P value
<0.01
<0.01
<0.01
<0.01
RI PT
Statistics
SC
Table 3-2
Table 3-2: MAIS reliability between evaluators (n=30) Overall
Self-report
Alerting to
Deriving
about the
Sounds
Meaning
M AN U
Statistics Value
Device 0.889
P value
<0.01
0.811
TE D
ICC value
<0.01
from Sounds
0.840
0.825
<0.01
<0.01
Overall
AC C
Statistics
EP
Table 3-3 MUSS reliability between evaluators (n=30)
Value
Sound
Communication Communication
Generation
Capability
Skills
ICC value
0.910
0.834
0.876
0.898
P value
<0.01
<0.01
<0.01
<0.01
Validity (1) Content validity Four experts believed that the content of the items of the three scales
ACCEPTED MANUSCRIPT
were in line with the measurement requirements, the representativeness of each item, and the content validity were acceptable. This means that the content validity of the 3 scales was good.
RI PT
(2) Construct validity Table 4-1 IT-MAIS Pearson Correlation Analysis（n=100） Vocalization Behavior
Vocalization Behavior Alerting to Sounds Deriving Meaning from Sounds
0.841
--
0.952
--
Deriving Meaning from Sounds --
0.809
--
--
0.685
0.812
--
TE D
0.943
Alerting to Sounds
SC
Overall
M AN U
Statistics
Table 4-2 MAIS Pearson Correlation Analysis（n=100） Overall
Alerting to Sounds
0.699
Self-report about the Device --
--
Deriving Meaning from Sounds --
0.904
0.612
--
--
0.930
0.516
0.712
--
EP
Statistics
AC C
Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
ACCEPTED MANUSCRIPT
Table 4-3 MUSS Pearson Correlation Analysis（n=100） Over all
Sound Generation
Sound
Communication
Communication
Generation
Capability
Skills
--
--
--
0.851
--
0.693
0.838
0.910
Communication 0.978
--
--
M AN U
Communication 0.893 Skills
SC
Capability
RI PT
Statistics
Discussion
Children are undergoing cochlear implantation at younger and
TE D
younger ages. This decline in age and the challenges associated with accurately assessing the functional hearing of infants raises concerns regarding the tools used for CI candidacy evaluation and post-CI progress
EP
assessments. The three scales, IT-MAIS, MAIS and MUSS, have been
AC C
very widely used by audiologists and speech-language pathologists in our clinical work, they can provide benefits for deaf children, including diagnosis, setting treatment goals, and tracking progress. As caregiver report tools, parents can also measure listening and speaking skills in children at home fast and conveniently. However, they have not undergone rigorous psychometric analysis, this present study was undertaken to investigate the reliability and validity of these three scales.
ACCEPTED MANUSCRIPT
Reliability Reliability is the consistency or stability of a measure. A measure is
RI PT
considered to have a high reliability if it produces similar results under consistent conditions. Reliability is expressed by correlation coefficient which can be divided into the stability coefficient and internal
SC
consistency coefficient. [12]
M AN U
Reliability in stability coefficient consists of test-retest reliability and reliability between evaluators, measured by intraclass correlation coefficient (ICC). The ICC was first used in 1966 by Bartko for the measurement and evaluation of reliability of size. The value below 0.4
TE D
indicates poor reliability, greater than 0.75 indicate a good reliability. [10] Test-retest reliability was used to evaluate the same questionnaire for the same group of children to a certain time interval repeat evaluation.
EP
The three scales’ overall measure showed excellent test-retest reliability
AC C
(0.921-0.929) and the three domains of each scale showed moderate reliability (0.832-0.903). The results showed that all the three scales’ test-retest reliability correlation coefficient of the whole scale and 3 domains are greater than 0.750, the three scales have acceptable test-retest reliability. Reliability between evaluators evaluated the same subjects by different evaluators without discussion. In this assessment, the ICC value of the overall scale of IT-MAIS、MAIS and MUSS are
ACCEPTED MANUSCRIPT
0.894, 0.889 and 0.910, greater than 0.750.While, the three domains of the three scales are all acceptable. The three scales have good reliability between evaluators.
RI PT
Reliability in internal consistency is assessed by Cronbach’s α. It reflects a measure of the relationship among all the items. High Cronbach’s α suggests that items have a close relationship and they are all
SC
measuring different aspects of the same construct. An acceptable
M AN U
reliability should greater than 0.7 or above. The results shows that the three scales have an excellent internal consistency. Validity
Validity refers to the degree of correctness, that is, a scale can
TE D
accurately measure what it is intended to measure which include effectiveness and correctness of the scale. Validity is an important condition for a scientific test tool to be useful.
EP
Content validity is a subjective indicator which refers to the
AC C
measurement of content and compliance for the selected topic, measurement is consistent with the measurement objective and requirement. In this study, three scales of each item evaluation by experts, considering all the items are in compliance with the measurement objective and requirement, have good content validity. Construct validity refers to the questionnaire can measure the characteristics of the theoretically expected degree. Evaluation methods
ACCEPTED MANUSCRIPT
are factor analysis, internal consistency test etc. This study selected the internal consistency to test construct validity of the scale. It meets the requirements of psychometrics if the correlation coefficient of each
RI PT
domain was 0.3~0.5 and the correlation coefficient between each domain and the whole scale was 0.5~0.8. The analysis showed that the IT-MAIS scale 3 domains with the total scale of correlation were between
SC
0.841~0.952 and relationship between the 3 domains were among
M AN U
0.685~0.812, 3 domains with the total scale correlation is very good. There was some correlation among the 3 domains but the correlation was less that of the total scale. The same results occured in the other two scales.
TE D
As we can see in the results, some domains in the three scales correlation coefficients are much higher, this may be because children's auditory and speech development is a continuous process, there is a close between
EP
relationship
listening
development
and
language
AC C
development.[13]scales have certain continuity between each sub dimensions. For IT-MAIS and MAIS, the correlation coefficient of alerting to sounds and deriving meaning from sound is high. This is probably because the auditory development in infants is step by step, well listening ability are more likely to exhibit better understanding ability, good ability of understanding based on a better awareness, it has relevance itself, and may lead to high correlation coefficient between
ACCEPTED MANUSCRIPT
different domains. The MAIS, the self-report about the device has relatively low correlations with the whole scale and the other two domains, our consideration is that how they use their devices is children's
RI PT
habits, different children on device preferences and habits are different, so the relationship is relatively low, but still within the normal range. The MUSS measures infants and children's verbal ability. Its three domains
SC
are voice, communication capability and communication skills. It has a
M AN U
very strong continuity because children often first learn to say something, and then to communicate and use some skills. The MUSS also has strong correlation. In conclusion, the three scales have acceptable construct validity.
TE D
The results provide strong evidence that the three scales possess good reliability and appropriate validity, and can be used to measure the outcome of cochlear implant in children. Considering the limited sample
EP
size, we can expand the sample size for further discussion and research.
AC C
Acknowledgements
Thanks to all the children and their families who participated in the
study. We would like to gratefully acknowledge Wenfang Wu from Capital Medical University for the statistical analyses. The project was supported by the capital citizen health program to foster from Beijing Municipal Science & Technology Commission (No.Z141100002114033), the Beijing Municipal Administration of Hospitals Clinical medicine
ACCEPTED MANUSCRIPT
Development of special funding support (No.XMLX201514), the National Natural Science Foundation of China (No.81371103), the capital health research and development of special from the Beijing Municipal
RI PT
Health Bureau (No.2011-1017-01). Thanks for the hard work on the training center of Beijing artificial cochlear language and the training
SC
center of China and Australia. References
years:
M AN U
[1] Osberger MJ. Cochlear implantation in children under the age of two candidacy
considerations.
1997;117:145–9.
Otolaryngol
Head
Neck
Surg
[2] Valencia DM, Rimell FL, Friedman BJ, Oblander MR, Helmbrecht J.
TE D
Cochlear implantation in infants less than 12 months of age. Int J Pediatr Otorhinolaryngol 2008;72:767–73.
EP
[3]Robbins A M, Rnesh A W J J, Berry S W. Evaluating meaningful auditory integration in profoundly hearing impaired children [J].Am J
AC C
Otol,1991,12 Suppl:144~150. [4]Osberger M J, Geier L, Zimmerman Phillips S, et al. Use of a parent-report scale to assess benefit in children given the Clarion cochlear implant [J].Am J Otol,1997,18(6 Suppl):S79~80. [5]Weichbold V, Anderson H, Haese PD: Validation of three adaptations of the Meaningful Auditory Integration Scale (MAIS) to German, English and Polish. Int J Audiol 2004; 43: 156–161.
ACCEPTED MANUSCRIPT
[6] Yun Zheng, Sigfrid D. Soli, Kai Wang, et al. A normative study of early prelingual auditory development. Audiol Neurotol 2009;14:214– 222.
RI PT
[7] Anne Schubert.Examining the validity and reliability of the IT-MAIS via RASCH analysis. B.A.University of Michigan, 2013.
[8]Zimmerman-Phillips, S., Osberger, M. J., & Robbins, A. M. (2001).
M AN U
Advanced Bionics Corporation.
SC
Infant-Toddler Meaningful Auditory Integration Scale. Sylmar, CA:
[9]Xueqing Chen, Sha Liu, Bo Liu1, et al. The effects of age at cochlear implantation and hearing aid trial on auditory performance of Chinese infants. Acta Oto-Laryngologica, 2010; 130: 263–270.
TE D
[10]Bartko JJ. The intraclass co rrelation coefficient as a measure of reliability.P sycho l Rep, 1966; 19:3.
[11]Revelle. W., Zinbarg, R.E. Coefficients alpha, beta, omega, and the
EP
glb: Comments on Sijtsma.Psychometrika,2009,74:145~154.
AC C
[12]Hinderink JB, Krabbe PF, Van Den Broek P Development and application of a health-related quality-oflife instrument for adults with cochlear implants: the Nijmegen cochlear implant questionnaire,2000. [13]Arlinger, S., Lunner, T., Lyxell, B., & Pichora-Fuller, K., M. (2009). The emergence of cognitive hearing science. Scandinavian Journal of Psychology, 50, 371-384. doi: 10.1111/j.1467- 9450.2009.00753.
ACCEPTED MANUSCRIPT
Table 1-1: IT-MAIS Test-retest Reliability Coefficients (n=30) Statistics
Overall
Value
Vocalization
Alerting to
Deriving
Behavior
Sounds
Meaning
0.929
0.902
0.832
P value
<0.01
<0.01
<0.01
0.875 <0.01
SC
ICC value
RI PT
from Sounds
Statistics
M AN U
Table 1-2: MAIS Test-retest Reliability Coefficients (n=30) Overall
Value
Self-report
Alerting to
Deriving
about the
Sounds
Meaning
Device 0.921
P value
<0.01
0.903
TE D
ICC value
EP
<0.01
from Sounds
0.854
0.890
<0.01
<0.01
Table 1-3 MUSS Test-retest Reliability Coefficients (n=30) Overall
AC C
Statistics Value
Sound
Communication Communication
Generation
Capability
Skills
ICC value
0.928
0.893
0.900
0.898
P value
<0.01
<0.01
<0.01
<0.01
Table 2-1 IT-MAIS Cronbach’s α Coefficients（n=100） Statistics
Average
Overall
26.84
Standard Deviation 11.281
Cronbach’s α 0.836
ACCEPTED MANUSCRIPT
6.84
1.868
0.888
11.58
4.780
0.762
8.42
5.474
0.747
RI PT
Vocalization Behavior Alerting to Sounds Deriving Meaning from Sounds
Table 2-2 MAIS Cronbach’s α Coefficients（n=100） Average
Overall Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
28.07 6.99
Cronbach’s α 0.747 0.881
3.600
0.754
4.850
0.702
TE D
9.05
M AN U
12.03
Standard Deviation 8.757 1.418
SC
Statistics
EP
Table 2-3 MUSS Cronbach’s α Coefficients（n=100）
Overall Sound Generation
18.85 8.79
Standard Deviation 11.200 3.382
Communication
6.97
5.555
0.743
3.09
3.009
0.856
AC C
Statistics
Average
Cronbach’s α 0.886 0.839
Capability Communication Skills
Table 3-1 IT-MAIS reliability between evaluators（n=30）
ACCEPTED MANUSCRIPT
Overall
Vocalization Behavior
Alerting to Sounds
0.894
0.813
0.870
Deriving Meaning from Sounds 0.842
<0.01
<0.01
<0.01
<0.01
ICC value P value
RI PT
Statistics
Table 3-2
Statistics
Overall
Value
Self-report
Alerting to
Deriving
about the
SC
Table 3-2: MAIS reliability between evaluators (n=30)
Sounds
Meaning
M AN U
Device
from Sounds
0.889
0.811
0.840
0.825
P value
<0.01
<0.01
<0.01
<0.01
TE D
ICC value
Table 3-3 MUSS reliability between evaluators (n=30)
Value
0.910
AC C
ICC value
Overall
Sound
Communication Communication
Generation
Capability
Skills
0.834
0.876
0.898
<0.01
<0.01
<0.01
EP
Statistics
P value
<0.01
Table 4-1 IT-MAIS Pearson Correlation Analysis（n=100）
Statistics
Overall
Vocalization Behavior
Alerting to Sounds
Vocalization Behavior Alerting to
0.841
--
--
Deriving Meaning from Sounds --
0.952
0.809
--
--
ACCEPTED MANUSCRIPT
0.943
0.685
0.812
--
RI PT
Sounds Deriving Meaning from Sounds
Table 4-2 MAIS Pearson Correlation Analysis（n=100）
Self-report about the Device Alerting to Sounds Deriving Meaning from Sounds
0.699
Self-report about the Device --
Alerting to Sounds
SC
Overall
--
Deriving Meaning from Sounds --
M AN U
Statistics
0.904
--
--
0.516
0.712
--
TE D
0.930
0.612
Table 4-3 MUSS Pearson Correlation Analysis（n=100） Overa ll
Sound
Communication
Communication
Generation
Capability
Skills
--
--
--
0.851
--
--
0.693
0.838
--
AC C
EP
Statistics
Sound Generation
0.910
Communication 0.978 Capability Communication 0.893 Skills