- Email: [email protected]
Relationship between index of complexity, outcome and need, dental aesthetic index, peer assessment rating index, and American Board of Orthodontics objective grading system
ORIGINAL ARTICLE
Relationship between index of complexity, outcome and need, dental aesthetic index, peer assessment rating index, and American Board of Orthodontics objective grading system Chukwudi Ochi Onyeasoa and Ellen A Begoleb Ibadan, Nigeria, and Chicago, Ill Introduction: The objectives of this study were to examine the relationships among 4 indexes that are used to score orthodontic treatment need and outcome, and to determine whether 1 index could replace the other 3. The index of complexity, outcome, and need (ICON), the dental aesthetic index (DAI), the peer assessment rating (PAR) index, and the American Board of Orthodontics objective grading system (ABO-OGS) were studied. The use of the ICON is desirable because it is faster to use than separate indexes for assessing different treatment facets. Methods: One hundred pretreatment and posttreatment study models were randomly selected from an accredited graduate orthodontic clinic at University of Illinois at Chicago, Chicago, Ill. The sample was assessed by using the PAR index, the DAI, the ABO-OGS, and the ICON. The data were analyzed descriptively and the relationships evaluated by using the Spearman rank order correlation coefficient. Results: The study showed a significant correlation between the DAI and the ICON with respect to treatment need (r ⫽ .5523, P ⬍.001). The PAR index and the ABO-OGS had significant correlations with the ICON for treatment outcome (pretreatment PAR vs ICON, r ⫽ .6840, P ⬍.001; posttreatment PAR vs ICON, r ⫽ .3466, P ⬍.001; ABO-OGS vs ICON, r ⫽ .2654, P ⬍.01). Conclusions: The ICON can be used in place of the PAR and the ABO-OGS for assessing treatment outcome and in place of the DAI for assessing treatment need. (Am J Orthod Dentofacial Orthop 2007;131:248-52)
T
he usefulness of occlusal indexes in audit, research, decision making, and assessing orthodontic treatment need and outcome is well accepted internationally. The peer assessment rating (PAR) index was developed to provide a summary score for occlusal anomalies and an estimate of how far a malocclusion deviates from normal alignment and occlusion.1 It was weighted to match the judgment of a panel of British orthodontists and general dentists on the deviation of a case from normal1 and has been used to evaluate treatment standards.2-8 The dental aesthetic index (DAI) was designed primarily to provide objective measures of esthetics and associated psychological handicaps based on 10 occlusal traits evaluated from pretreatment study models.9,10 a
Lecturer/consultant orthodontist, Department of Preventive Dentistry, University of Ibadan, Ibadan, Nigeria. b Associate professor, Department of Orthodontics, University of Illinois at Chicago. Reprint requests to: Chukwudi Ochi Onyeaso, Department of Child Oral Health, Faculty of Dentistry, College of Medicine, University of Ibadan, Ibadan, Nigeria; e-mail, [email protected]. Submitted, February 2005; revised and accepted, April 2005. 0889-5406/$32.00 Copyright © 2007 by the American Association of Orthodontists. doi:10.1016/j.ajodo.2005.04.045
248
It is an orthodontic index for treatment need assessment. Although the DAI was initially based on dental records of white adolescents in the United States,9-13 it was accepted by the World Health Organization as a cross-cultural international index.14-16 Despite the various advantages of DAI, such as high reliability and validity,14 it has limitations; eg, it excludes missing molars, impacted teeth, posterior crossbites, and midline discrepancies from the computation of scores. Therefore, its esthetic assessment is believed to be limited by these exclusions.17 In 1994, the American Board of Orthodontics (ABO) began investigating methods of making the phase III examination more objective.18 An objective method of evaluating dental casts and intraoral radiographs was eventually developed through a series of 4 field tests over 5 years.18 In 1999, the ABO instituted the model and radiographic portions of the objective grading system (OGS) to be officially used to grade these portions of candidates’ clinical case reports. The ABO’s OGS (ABO-OGS) attempts to assess the outcome of orthodontic treatment. To address the issues of treatment need complexity, treatment improvement, and outcome based on interna-
Onyeaso and Begole 249
American Journal of Orthodontics and Dentofacial Orthopedics Volume 131, Number 2
Table I.
Intraexaminer reliability for 4 orthodontic indexes ICON
RMS (random error) Spearman rank order correlation coefficient
PAR
Pretreatment
Posttreatment
Pretreatment
Posttreatment
DAI
ABO-OGS
.16 .99*
.00 1.000*
1.24 .99*
.10 .99*
.43 .93*
.07 .99*
*P ⬍.001.
tional professional opinions, intended for use in the context of specialist practice, the index of complexity, outcome, and need (ICON) was developed.19 This index could provide the means to compare treatment thresholds in various countries and serve as the basis for quality assurance standards in orthodontics.19 The ICON was developed in 9 countries by 97 orthodontists and is arguably more valid than the PAR index.19,20 High validity of the ICON has been reported,21 and several European studies also showed good reliability.20,22,23 This index appears to be easier and faster to use than separate indexes for various facets of orthodontic treatment. The indexes used in this study assessed need only (DAI), outcome only (PAR and ABO-OGS), and both need and outcome (ICON). Because of the multifactorial nature of malocclusion, which includes a patient’s expectations and psychological needs, and physical characteristics of the occlusion,24,25 it is difficult to standardize judgments.26 Therefore, our purposes in this study were to (1) assess the relationship between the ICON and the PAR, the DAI, and the ABO-OGS; and (2) establish whether the ICON can replace these indexes as a measure of orthodontic treatment complexity, outcome, and need instead of using different indexes for the various facets of orthodontic treatment. MATERIAL AND METHODS
A retrospective sample of 100 treated patients was randomly selected from an accredited graduate clinic— Department of Orthodontics, University of Illinois at Chicago, Chicago, Ill. The patients were treated by graduate orthodontic residents under the supervision of qualified orthodontic specialists who were full-time or part-time faculty members in the orthodontic department. Pretreatment and posttreatment study models were scored with the PAR1 and the ICON.19 Only pretreatment models were scored with the DAI,9,14 and posttreatment models were scored according to the ABOOGS by a calibrated examiner (C.O.O.) from outside the United States. The radiographic aspect in the
ABO-OGS was not included. The examiner strictly followed the procedures for each index. Although the exact time taken to assess the casts with each index was not strictly recorded, the examiner’s experience during the exercise showed that the ICON was the fastest to use, with about 3 minutes spent for each set of casts, whereas the PAR had the longest duration. Intraexaminer reliability was assessed by repeating the examination of a random subset of 20 pretreatment and posttreatment casts 4 weeks after the initial examinations. Data analysis
Intraexaminer reliability was tested by using the root mean square (RMS)27 for random error and further with the Spearman rank order correlation coefficient. In addition to the descriptive statistics, the relationship between the ICON and the other indexes was examined by using nonparametric (Spearman) correlation. The correlation was calculated separately for the pretreatment and posttreatment data. Occlusal improvement with the ICON is assessed by subtracting 4 times the posttreatment score from the pretreatment score.19 There are 5 categories of improvement. Outcome is assessed with either the percentage change or the absolute change in occlusal index scores. The 5 categories of improvement in the ICON can be considered similar to the 3 PAR improvement grades as determined from the PAR monogram.28 RESULTS
Table I shows the reliability assessments of the indexes with RMS for random error and with the Spearman rank order correlation coefficient. Descriptive statistics for the indexes are given in Table II. The pretreatment mean ICON was 67.8 ⫾ 20.6 (SD); the posttreatment mean ICON was 15.1 ⫾ 2.9 (SD). Pretreatment and posttreatment weighted PAR scores were 23.8 ⫾ 11.5 (SD) and 1.7 ⫾ 3.8 (SD), respectively. Both the ICON and the PAR had higher pretreatment scores, as expected.
250 Onyeaso and Begole
American Journal of Orthodontics and Dentofacial Orthopedics February 2007
Table II.
Descriptive statistics for entire sample for ICON, PAR, DAI, and ABO-OGS
120
100
Index ICON DAI PAR Weighted Unweighted ABO-OGS
Pretreatment Mean (SD)
Posttreatment Mean (SD)
67.8 (20.6) 36.3 (13.4) — 23.8 (11.5) 18.6 (9.3) —
15.1 (2.9) — — 1.7 (3.8) 0.96 (1.8) 16.4 (6.8)
80
60
40
20
0
Table III.
Spearman rank order correlation coefficients for ICON, PAR, DAI, and ABO-OGS
Spearman correlations between indexes
Pretreatment
Posttreatment
†
ICON vs DAI ICON vs PAR ICON vs ABO-OGS PAR vs ABO-OGS
— .3466† .2654* .2755*
.5523 .6840† — —
-10
0
10
20
30
40
50
60
– ڤtotal pre-treatment ICON score versus pre-treatment weighted PAR score X – total post-treatment ICON score versus post-treatment weighted PAR score
Fig 2. Scatterplot showing total pretreatment ICON score vs pretreatment weighted PAR score and total posttreatment ICON score vs posttreatment weighted PAR score. 120
*P ⬍.01;†P ⬍.001.
% weighted PAR Reduction
100
total pre-treatment ICON score
120
100
80
60
80
60
40
20
0
-20 -200
40
-100
0
100
Degree of Improvement 20 20
30
40
50
60
70
total DAI score
Fig 1. Scatterplot showing total DAI score vs pretreatment ICON score.
Table III shows the correlation between ICON, DAI, and ABO-OGS scores. The highest correlation was found between pretreatment ICON vs PAR scores (r ⫽ .6840, P ⬍.001), followed by ICON vs DAI (r ⫽ .5523, P ⫽ .001). All indexes showed statistically significant positive correlations with the ICON. The distribution of orthodontic treatment needs an outcome for the study sample according to 4 indexes as shown in Table IV. Figure 1 shows the plot of the DAI vs the ICON (pretreatment). Definite treatment need starts at DAI
Fig 3. Scatterplot showing degree of ICON improvement vs percentage weighted PAR score reduction.
scores of 26 and above, whereas treatment need starts at ICON score greater than 43. This shows good agreement between ICON and DAI scores. Figure 2 is the plot of PAR scores against ICON scores for pretreatment and posttreatment study casts. Good agreement between the ICON and the PAR is shown. The degree of ICON improvement (pretreatment ICON score minus 4 times the posttreatment ICON score) vs percentage of PAR reduction is shown in Figure 3. A plot of posttreatment ICON scores vs ABO-OGS scores is shown in Figure 4.
Onyeaso and Begole 251
American Journal of Orthodontics and Dentofacial Orthopedics Volume 131, Number 2
Table IV. Distribution of orthodontic treatment need and outcome for sample (n ⫽ 100) according to 4 indexes
total pre-treatment ICON score
120
100
Treatment need Index
80
ICON DAI PAR ABO-OGS
60
Treatment outcome
Present
Absent
Acceptable
Unacceptable
86 85 — —
14 15 — —
94 — 97 86
6 — 3 14
40
20 0
10
20
30
40
ABOGS score
Fig 4. Scatterplot showing total posttreatment ICON score vs ABO-OGS score. DISCUSSION
The intraexaminer reliability assessment outcome in this study is considered satisfactory and is comparable to several related previous reports.20,22,26,27 Indexes that assess only outcome of orthodontic treatment, such as the ABO-OGS, often do not seem to consider the severity of the pretreatment need or the complexity of the treatment. Those assessing need only, such as the DAI, often do not offer information on the complexity of the treatment. The ICON (assessing need, outcome, and complexity) seems to provide answers to these shortcomings. Using different indexes to assess need and outcome to some extent implies that malocclusion is not a continuum and that, after treatment, the occlusion cannot be considered to need further treatment. Using the same assessment before and after treatment seems to be more valid. The sample size in this study is an improvement on that used in a similar study.28 It also had a fairly broad range of treatment starts as did the previous study, but a major difference between these studies was that all patients in this American study were treated by graduate orthodontic residents under the supervision of qualified orthodontic specialists; in the United Kingdom (UK) study, some patients were treated by clinical assistants.28 Also, even though the IOTN was used for treatment need whereas the DAI was used in this study, the relationship between the ABO-OGS and the ICON was also assessed. The UK study included patients who did not complete treatments in line with normal orthodontic treatment protocol, whereas our study did not.28 The pretreatment mean PAR score in the UK study28 was 38.2 ⫾ 10.6 (SD) compared with 23.8 ⫾
11.5 (SD) in this American study. The posttreatment mean PAR score was 5.4 ⫾ 5.9 (SD) in the UK study; in this study, it was 1.7 ⫾ 3.8 (SD). The pretreatment ICON mean score was 67.8 ⫾ 20.6 (S.D) in our study compared with 72.8 ⫾ 13.0 (SD) in the UK.28 The results of the correlation between ICON and the other indexes showed that pretreatment models had higher correlations than posttreatment models. This agrees with the UK study.28 We found significantly high correlations between pretreatment ICON and PAR scores. We found the highest correlation between the ICON and the PAR. However, the degree of correlation (r ⫽ 0.55, P ⬍.01) between the ICON and the DAI (pretreatment) in this study is comparable with that between the ICON and the PAR reported in the UK (r ⫽ 0.51).17 If there is little variation in a sample, poor correlations result.27 Lower levels of correlations were generally obtained in this study for the posttreatment casts (probably because the treatment outcomes were consistently good). In assessing treatment need, the general performance of the ICON seemed to generally agree with the DAI. In our study, 14 patients had no orthodontic treatment need as assessed by the ICON compared with 15 according the DAI (Table IV). This suggests that the ICON could probably substitute for the DAI and produce similar results. The general performance of the ICON was also comparable with the PAR and the ABO-OGS because 6 and 3 patients had unacceptable results, according to the ICON and the PAR, respectively. The ICON was shown to be more stringent in assessing outcome than the PAR in this study. This supports the findings of Fox et al.28 Fourteen patients had unacceptable outcomes according to the ABOOGS. It would seem that the ABO-OGS is the most stringent in assessing treatment outcomes compared with the ICON and the PAR.
252 Onyeaso and Begole
CONCLUSIONS ●
●
●
●
The ICON can substitute for the DAI to measure orthodontic treatment need. A value greater than 43 for the ICON defines need for treatment, as does a DAI score of 26 or above. The relationship between these 2 indexes was statistically significant (P ⬍.001). The PAR had a close relationship with the ICON in this study; thus, the ICON can be used to assess orthodontic treatment outcome. The ABO-OGS requires more stringent standards than the PAR or the ICON for assessing the outcome of orthodontic treatment. Overall agreement between the ICON and the other indexes assessed in this study was good. Therefore, the ICON appears to be a reasonable means of assessing the standard of orthodontic treatment in terms of complexity, need, and outcome rather than using various indexes. Use of the ICON will encourage international comparison and professional standardization.
REFERENCES 1. Richmond S, Shaw WC, O’Brien KD, Buchanan IB, Jones R, Stephens CD, et al. The development of the PAR index (peer assessment rating): reliability and validity. Eur J Orthod 1992; 14:125-39. 2. Richmond S, Shaw WC, Stephens CD, Webb WG, Roberts CT, Andrews M. Orthodontics in the general dental service of England and Wales: a critical assessment of standards. Br Dent J 1993;174:315-29. 3. O’Brien KD, Shaw WC, Roberts CT. The use of occlusal indices in assessing the provision of orthodontic service of England and Wales. Br J Orthod 1993;20:25-35. 4. Richmond S, Shaw WC, Roberts CT, Andrews M. The PAR index (peer assessment rating): methods to determine outcome of orthodontic treatment in terms of improvement and standards. Eur J Orthod 1992;14:180-7. 5. Buchanan IB, Russell J, Clark JD. Practical application of the PAR index: an illustrative comparison of the outcome of treatment using two fixed appliance techniques. Br J Orthod 1996; 23:351-7. 6. Kerr WJS, Buchanan IB, McColl JH. Use of the PAR index in assessing the effectiveness of removable orthodontic appliances. Br J Orthod 1993;20:351-7. 7. Kerr WJS, Buchanan IB, McNair FI, McColl JH. Factors influencing the outcome and duration of removable appliance treatment. Eur J Orthod 1994;16:181-6. 8. Birkeland K, Furevik J, Boe OE, Wisth PJ. Evaluation of treatment and post-treatment changes by the PAR index. Eur J Orthod 1997;19:279-88. 9. Cons NC, Jenny J, Kohout FJ. DAI: the dental aesthetic index. Iowa City: College of Dentistry, University of Iowa; 1986.
American Journal of Orthodontics and Dentofacial Orthopedics February 2007
10. Cons NC, Mruthyunjaya YC, Pollard ST. Distribution of occlusal traits in a sample of 1,337 children aged 15-18 residing in upstate New York. Int Dent J 1978;28:154-64. 11. Proshek JM, Jenny J, Cons NC, Frazier PJ, Summers CJ, Mruthyunjaya YC. A factor analysis of traits as measured by COCSTOC-MOT to identify occlusion trait combinations. Community Dent Oral Epidemiol 1979;7:274-82. 12. Ast DB, Carlos JP, Cons NC. The prevalence and characteristics of malocclusion among senior high school students in upstate New York. Am J Orthod 1965;51:437-45. 13. Jenny J, Cons NC, Kohout FJ, Frazier PJ. Test of a method to determine socially acceptable occlusal conditions. Community Dent Oral Epidemiol 1980;8:424-33. 14. World Health Organization International Collaborative Study of Oral Health Outcomes (ICS II), document 2: oral data collection instrument and evaluative criteria. Geneva: World Health Organization; 1989. 15. Beglin FM, Firestone AR, Vig KW, Beck FM, Kuthy RA, Wade D. A comparison of the reliability and validity of 3 occlusal indexes of orthodontic treatment need. Am J Orthod Dentofacial Orthop 2001;120:240-6. 16. Cons NC, Jenny J. Comparing perceptions of dental aesthetics in the USA with those in eleven ethnic groups. Int Dent J 1994;44: 489-94. 17. Danyluk K, Lavelle C, Hassard T. Potential application of the dental aesthetic index to prioritize the orthodontic service needs in a publicly funded dental program. Am J Orthod Dentofacial Orthop 1999;116;279-86. 18. The American Board of Orthodontics. ABO objective grading system for dental casts and panoramic radiographs. Available at: www.americanboardortho.com/professionals/road to cert/phase III/Grading System 2004. Accessed February 2005. 19. Daniels C, Richmond S. the development of the index of complexity, outcome and need (ICON). J Orthod 2000;27: 149-62. 20. Richmond S, Ikonomou C, Williams B, Ramel S, Rolfe B, Kurol J. Orthodontic treatment standards in a public group practice in Sweden. Swed Dent J 2001;25:137-44. 21. Firestone AR, Beck M, Beglin FM, Vig KWL. Validity of the index of complexity, outcome, and need. Angle Orthod 2002;72: 15-20. 22. Richmond S, Ikonomou C, Williams B, Rolfe B. Orthodontic treatment standards in Greece. Hel Orthod Rev 2001;4:9-20. 23. Liepa A, Urtane I, Richmond S, Dunstan F. Orthodontic treatment need in Latvia. Eur J Orthod 2000;25:279-84. 24. Baume LJ, Marechaux S. Uniform methods for the epidemiologic assessment of malocclusion. Am J Orthod 1974;66:121-9. 25. McLain JB, Proffit WR. Oral health status in the United States: prevalence of malocclusion. J Dent Educ 1985;49:386-96. 26. Koochek A-R, Yeh MS, Rolfe B, Richmond S. The relationship between index of complexity, outcome and need, and patients’ perceptions of malocclusion: a study in general dental practice. Br Dent J 2001;191:325-9. 27. Roberts CT, Richmond S. The design and analysis of reliability studies for the use of epidemiological and audit indices in orthodontics. Br J Orthod 1997;24:139-47. 28. Fox NA, Daniels C, Gilgrass T. A comparison of the index of complexity, outcome and need (ICON) with the peer assessment rating (PAR) and the index of orthodontic treatment need (IOTN). Br Dent J 2002;193:225-30.
Relationship between index of complexity, outcome and need, dental aesthetic index, peer assessment rating index, and American Board of Orthodontics objective grading system Chukwudi Ochi Onyeasoa and Ellen A Begoleb Ibadan, Nigeria, and Chicago, Ill Introduction: The objectives of this study were to examine the relationships among 4 indexes that are used to score orthodontic treatment need and outcome, and to determine whether 1 index could replace the other 3. The index of complexity, outcome, and need (ICON), the dental aesthetic index (DAI), the peer assessment rating (PAR) index, and the American Board of Orthodontics objective grading system (ABO-OGS) were studied. The use of the ICON is desirable because it is faster to use than separate indexes for assessing different treatment facets. Methods: One hundred pretreatment and posttreatment study models were randomly selected from an accredited graduate orthodontic clinic at University of Illinois at Chicago, Chicago, Ill. The sample was assessed by using the PAR index, the DAI, the ABO-OGS, and the ICON. The data were analyzed descriptively and the relationships evaluated by using the Spearman rank order correlation coefficient. Results: The study showed a significant correlation between the DAI and the ICON with respect to treatment need (r ⫽ .5523, P ⬍.001). The PAR index and the ABO-OGS had significant correlations with the ICON for treatment outcome (pretreatment PAR vs ICON, r ⫽ .6840, P ⬍.001; posttreatment PAR vs ICON, r ⫽ .3466, P ⬍.001; ABO-OGS vs ICON, r ⫽ .2654, P ⬍.01). Conclusions: The ICON can be used in place of the PAR and the ABO-OGS for assessing treatment outcome and in place of the DAI for assessing treatment need. (Am J Orthod Dentofacial Orthop 2007;131:248-52)
T
he usefulness of occlusal indexes in audit, research, decision making, and assessing orthodontic treatment need and outcome is well accepted internationally. The peer assessment rating (PAR) index was developed to provide a summary score for occlusal anomalies and an estimate of how far a malocclusion deviates from normal alignment and occlusion.1 It was weighted to match the judgment of a panel of British orthodontists and general dentists on the deviation of a case from normal1 and has been used to evaluate treatment standards.2-8 The dental aesthetic index (DAI) was designed primarily to provide objective measures of esthetics and associated psychological handicaps based on 10 occlusal traits evaluated from pretreatment study models.9,10 a
Lecturer/consultant orthodontist, Department of Preventive Dentistry, University of Ibadan, Ibadan, Nigeria. b Associate professor, Department of Orthodontics, University of Illinois at Chicago. Reprint requests to: Chukwudi Ochi Onyeaso, Department of Child Oral Health, Faculty of Dentistry, College of Medicine, University of Ibadan, Ibadan, Nigeria; e-mail, [email protected]. Submitted, February 2005; revised and accepted, April 2005. 0889-5406/$32.00 Copyright © 2007 by the American Association of Orthodontists. doi:10.1016/j.ajodo.2005.04.045
248
It is an orthodontic index for treatment need assessment. Although the DAI was initially based on dental records of white adolescents in the United States,9-13 it was accepted by the World Health Organization as a cross-cultural international index.14-16 Despite the various advantages of DAI, such as high reliability and validity,14 it has limitations; eg, it excludes missing molars, impacted teeth, posterior crossbites, and midline discrepancies from the computation of scores. Therefore, its esthetic assessment is believed to be limited by these exclusions.17 In 1994, the American Board of Orthodontics (ABO) began investigating methods of making the phase III examination more objective.18 An objective method of evaluating dental casts and intraoral radiographs was eventually developed through a series of 4 field tests over 5 years.18 In 1999, the ABO instituted the model and radiographic portions of the objective grading system (OGS) to be officially used to grade these portions of candidates’ clinical case reports. The ABO’s OGS (ABO-OGS) attempts to assess the outcome of orthodontic treatment. To address the issues of treatment need complexity, treatment improvement, and outcome based on interna-
Onyeaso and Begole 249
American Journal of Orthodontics and Dentofacial Orthopedics Volume 131, Number 2
Table I.
Intraexaminer reliability for 4 orthodontic indexes ICON
RMS (random error) Spearman rank order correlation coefficient
PAR
Pretreatment
Posttreatment
Pretreatment
Posttreatment
DAI
ABO-OGS
.16 .99*
.00 1.000*
1.24 .99*
.10 .99*
.43 .93*
.07 .99*
*P ⬍.001.
tional professional opinions, intended for use in the context of specialist practice, the index of complexity, outcome, and need (ICON) was developed.19 This index could provide the means to compare treatment thresholds in various countries and serve as the basis for quality assurance standards in orthodontics.19 The ICON was developed in 9 countries by 97 orthodontists and is arguably more valid than the PAR index.19,20 High validity of the ICON has been reported,21 and several European studies also showed good reliability.20,22,23 This index appears to be easier and faster to use than separate indexes for various facets of orthodontic treatment. The indexes used in this study assessed need only (DAI), outcome only (PAR and ABO-OGS), and both need and outcome (ICON). Because of the multifactorial nature of malocclusion, which includes a patient’s expectations and psychological needs, and physical characteristics of the occlusion,24,25 it is difficult to standardize judgments.26 Therefore, our purposes in this study were to (1) assess the relationship between the ICON and the PAR, the DAI, and the ABO-OGS; and (2) establish whether the ICON can replace these indexes as a measure of orthodontic treatment complexity, outcome, and need instead of using different indexes for the various facets of orthodontic treatment. MATERIAL AND METHODS
A retrospective sample of 100 treated patients was randomly selected from an accredited graduate clinic— Department of Orthodontics, University of Illinois at Chicago, Chicago, Ill. The patients were treated by graduate orthodontic residents under the supervision of qualified orthodontic specialists who were full-time or part-time faculty members in the orthodontic department. Pretreatment and posttreatment study models were scored with the PAR1 and the ICON.19 Only pretreatment models were scored with the DAI,9,14 and posttreatment models were scored according to the ABOOGS by a calibrated examiner (C.O.O.) from outside the United States. The radiographic aspect in the
ABO-OGS was not included. The examiner strictly followed the procedures for each index. Although the exact time taken to assess the casts with each index was not strictly recorded, the examiner’s experience during the exercise showed that the ICON was the fastest to use, with about 3 minutes spent for each set of casts, whereas the PAR had the longest duration. Intraexaminer reliability was assessed by repeating the examination of a random subset of 20 pretreatment and posttreatment casts 4 weeks after the initial examinations. Data analysis
Intraexaminer reliability was tested by using the root mean square (RMS)27 for random error and further with the Spearman rank order correlation coefficient. In addition to the descriptive statistics, the relationship between the ICON and the other indexes was examined by using nonparametric (Spearman) correlation. The correlation was calculated separately for the pretreatment and posttreatment data. Occlusal improvement with the ICON is assessed by subtracting 4 times the posttreatment score from the pretreatment score.19 There are 5 categories of improvement. Outcome is assessed with either the percentage change or the absolute change in occlusal index scores. The 5 categories of improvement in the ICON can be considered similar to the 3 PAR improvement grades as determined from the PAR monogram.28 RESULTS
Table I shows the reliability assessments of the indexes with RMS for random error and with the Spearman rank order correlation coefficient. Descriptive statistics for the indexes are given in Table II. The pretreatment mean ICON was 67.8 ⫾ 20.6 (SD); the posttreatment mean ICON was 15.1 ⫾ 2.9 (SD). Pretreatment and posttreatment weighted PAR scores were 23.8 ⫾ 11.5 (SD) and 1.7 ⫾ 3.8 (SD), respectively. Both the ICON and the PAR had higher pretreatment scores, as expected.
250 Onyeaso and Begole
American Journal of Orthodontics and Dentofacial Orthopedics February 2007
Table II.
Descriptive statistics for entire sample for ICON, PAR, DAI, and ABO-OGS
120
100
Index ICON DAI PAR Weighted Unweighted ABO-OGS
Pretreatment Mean (SD)
Posttreatment Mean (SD)
67.8 (20.6) 36.3 (13.4) — 23.8 (11.5) 18.6 (9.3) —
15.1 (2.9) — — 1.7 (3.8) 0.96 (1.8) 16.4 (6.8)
80
60
40
20
0
Table III.
Spearman rank order correlation coefficients for ICON, PAR, DAI, and ABO-OGS
Spearman correlations between indexes
Pretreatment
Posttreatment
†
ICON vs DAI ICON vs PAR ICON vs ABO-OGS PAR vs ABO-OGS
— .3466† .2654* .2755*
.5523 .6840† — —
-10
0
10
20
30
40
50
60
– ڤtotal pre-treatment ICON score versus pre-treatment weighted PAR score X – total post-treatment ICON score versus post-treatment weighted PAR score
Fig 2. Scatterplot showing total pretreatment ICON score vs pretreatment weighted PAR score and total posttreatment ICON score vs posttreatment weighted PAR score. 120
*P ⬍.01;†P ⬍.001.
% weighted PAR Reduction
100
total pre-treatment ICON score
120
100
80
60
80
60
40
20
0
-20 -200
40
-100
0
100
Degree of Improvement 20 20
30
40
50
60
70
total DAI score
Fig 1. Scatterplot showing total DAI score vs pretreatment ICON score.
Table III shows the correlation between ICON, DAI, and ABO-OGS scores. The highest correlation was found between pretreatment ICON vs PAR scores (r ⫽ .6840, P ⬍.001), followed by ICON vs DAI (r ⫽ .5523, P ⫽ .001). All indexes showed statistically significant positive correlations with the ICON. The distribution of orthodontic treatment needs an outcome for the study sample according to 4 indexes as shown in Table IV. Figure 1 shows the plot of the DAI vs the ICON (pretreatment). Definite treatment need starts at DAI
Fig 3. Scatterplot showing degree of ICON improvement vs percentage weighted PAR score reduction.
scores of 26 and above, whereas treatment need starts at ICON score greater than 43. This shows good agreement between ICON and DAI scores. Figure 2 is the plot of PAR scores against ICON scores for pretreatment and posttreatment study casts. Good agreement between the ICON and the PAR is shown. The degree of ICON improvement (pretreatment ICON score minus 4 times the posttreatment ICON score) vs percentage of PAR reduction is shown in Figure 3. A plot of posttreatment ICON scores vs ABO-OGS scores is shown in Figure 4.
Onyeaso and Begole 251
American Journal of Orthodontics and Dentofacial Orthopedics Volume 131, Number 2
Table IV. Distribution of orthodontic treatment need and outcome for sample (n ⫽ 100) according to 4 indexes
total pre-treatment ICON score
120
100
Treatment need Index
80
ICON DAI PAR ABO-OGS
60
Treatment outcome
Present
Absent
Acceptable
Unacceptable
86 85 — —
14 15 — —
94 — 97 86
6 — 3 14
40
20 0
10
20
30
40
ABOGS score
Fig 4. Scatterplot showing total posttreatment ICON score vs ABO-OGS score. DISCUSSION
The intraexaminer reliability assessment outcome in this study is considered satisfactory and is comparable to several related previous reports.20,22,26,27 Indexes that assess only outcome of orthodontic treatment, such as the ABO-OGS, often do not seem to consider the severity of the pretreatment need or the complexity of the treatment. Those assessing need only, such as the DAI, often do not offer information on the complexity of the treatment. The ICON (assessing need, outcome, and complexity) seems to provide answers to these shortcomings. Using different indexes to assess need and outcome to some extent implies that malocclusion is not a continuum and that, after treatment, the occlusion cannot be considered to need further treatment. Using the same assessment before and after treatment seems to be more valid. The sample size in this study is an improvement on that used in a similar study.28 It also had a fairly broad range of treatment starts as did the previous study, but a major difference between these studies was that all patients in this American study were treated by graduate orthodontic residents under the supervision of qualified orthodontic specialists; in the United Kingdom (UK) study, some patients were treated by clinical assistants.28 Also, even though the IOTN was used for treatment need whereas the DAI was used in this study, the relationship between the ABO-OGS and the ICON was also assessed. The UK study included patients who did not complete treatments in line with normal orthodontic treatment protocol, whereas our study did not.28 The pretreatment mean PAR score in the UK study28 was 38.2 ⫾ 10.6 (SD) compared with 23.8 ⫾
11.5 (SD) in this American study. The posttreatment mean PAR score was 5.4 ⫾ 5.9 (SD) in the UK study; in this study, it was 1.7 ⫾ 3.8 (SD). The pretreatment ICON mean score was 67.8 ⫾ 20.6 (S.D) in our study compared with 72.8 ⫾ 13.0 (SD) in the UK.28 The results of the correlation between ICON and the other indexes showed that pretreatment models had higher correlations than posttreatment models. This agrees with the UK study.28 We found significantly high correlations between pretreatment ICON and PAR scores. We found the highest correlation between the ICON and the PAR. However, the degree of correlation (r ⫽ 0.55, P ⬍.01) between the ICON and the DAI (pretreatment) in this study is comparable with that between the ICON and the PAR reported in the UK (r ⫽ 0.51).17 If there is little variation in a sample, poor correlations result.27 Lower levels of correlations were generally obtained in this study for the posttreatment casts (probably because the treatment outcomes were consistently good). In assessing treatment need, the general performance of the ICON seemed to generally agree with the DAI. In our study, 14 patients had no orthodontic treatment need as assessed by the ICON compared with 15 according the DAI (Table IV). This suggests that the ICON could probably substitute for the DAI and produce similar results. The general performance of the ICON was also comparable with the PAR and the ABO-OGS because 6 and 3 patients had unacceptable results, according to the ICON and the PAR, respectively. The ICON was shown to be more stringent in assessing outcome than the PAR in this study. This supports the findings of Fox et al.28 Fourteen patients had unacceptable outcomes according to the ABOOGS. It would seem that the ABO-OGS is the most stringent in assessing treatment outcomes compared with the ICON and the PAR.
252 Onyeaso and Begole
CONCLUSIONS ●
●
●
●
The ICON can substitute for the DAI to measure orthodontic treatment need. A value greater than 43 for the ICON defines need for treatment, as does a DAI score of 26 or above. The relationship between these 2 indexes was statistically significant (P ⬍.001). The PAR had a close relationship with the ICON in this study; thus, the ICON can be used to assess orthodontic treatment outcome. The ABO-OGS requires more stringent standards than the PAR or the ICON for assessing the outcome of orthodontic treatment. Overall agreement between the ICON and the other indexes assessed in this study was good. Therefore, the ICON appears to be a reasonable means of assessing the standard of orthodontic treatment in terms of complexity, need, and outcome rather than using various indexes. Use of the ICON will encourage international comparison and professional standardization.
REFERENCES 1. Richmond S, Shaw WC, O’Brien KD, Buchanan IB, Jones R, Stephens CD, et al. The development of the PAR index (peer assessment rating): reliability and validity. Eur J Orthod 1992; 14:125-39. 2. Richmond S, Shaw WC, Stephens CD, Webb WG, Roberts CT, Andrews M. Orthodontics in the general dental service of England and Wales: a critical assessment of standards. Br Dent J 1993;174:315-29. 3. O’Brien KD, Shaw WC, Roberts CT. The use of occlusal indices in assessing the provision of orthodontic service of England and Wales. Br J Orthod 1993;20:25-35. 4. Richmond S, Shaw WC, Roberts CT, Andrews M. The PAR index (peer assessment rating): methods to determine outcome of orthodontic treatment in terms of improvement and standards. Eur J Orthod 1992;14:180-7. 5. Buchanan IB, Russell J, Clark JD. Practical application of the PAR index: an illustrative comparison of the outcome of treatment using two fixed appliance techniques. Br J Orthod 1996; 23:351-7. 6. Kerr WJS, Buchanan IB, McColl JH. Use of the PAR index in assessing the effectiveness of removable orthodontic appliances. Br J Orthod 1993;20:351-7. 7. Kerr WJS, Buchanan IB, McNair FI, McColl JH. Factors influencing the outcome and duration of removable appliance treatment. Eur J Orthod 1994;16:181-6. 8. Birkeland K, Furevik J, Boe OE, Wisth PJ. Evaluation of treatment and post-treatment changes by the PAR index. Eur J Orthod 1997;19:279-88. 9. Cons NC, Jenny J, Kohout FJ. DAI: the dental aesthetic index. Iowa City: College of Dentistry, University of Iowa; 1986.
American Journal of Orthodontics and Dentofacial Orthopedics February 2007
10. Cons NC, Mruthyunjaya YC, Pollard ST. Distribution of occlusal traits in a sample of 1,337 children aged 15-18 residing in upstate New York. Int Dent J 1978;28:154-64. 11. Proshek JM, Jenny J, Cons NC, Frazier PJ, Summers CJ, Mruthyunjaya YC. A factor analysis of traits as measured by COCSTOC-MOT to identify occlusion trait combinations. Community Dent Oral Epidemiol 1979;7:274-82. 12. Ast DB, Carlos JP, Cons NC. The prevalence and characteristics of malocclusion among senior high school students in upstate New York. Am J Orthod 1965;51:437-45. 13. Jenny J, Cons NC, Kohout FJ, Frazier PJ. Test of a method to determine socially acceptable occlusal conditions. Community Dent Oral Epidemiol 1980;8:424-33. 14. World Health Organization International Collaborative Study of Oral Health Outcomes (ICS II), document 2: oral data collection instrument and evaluative criteria. Geneva: World Health Organization; 1989. 15. Beglin FM, Firestone AR, Vig KW, Beck FM, Kuthy RA, Wade D. A comparison of the reliability and validity of 3 occlusal indexes of orthodontic treatment need. Am J Orthod Dentofacial Orthop 2001;120:240-6. 16. Cons NC, Jenny J. Comparing perceptions of dental aesthetics in the USA with those in eleven ethnic groups. Int Dent J 1994;44: 489-94. 17. Danyluk K, Lavelle C, Hassard T. Potential application of the dental aesthetic index to prioritize the orthodontic service needs in a publicly funded dental program. Am J Orthod Dentofacial Orthop 1999;116;279-86. 18. The American Board of Orthodontics. ABO objective grading system for dental casts and panoramic radiographs. Available at: www.americanboardortho.com/professionals/road to cert/phase III/Grading System 2004. Accessed February 2005. 19. Daniels C, Richmond S. the development of the index of complexity, outcome and need (ICON). J Orthod 2000;27: 149-62. 20. Richmond S, Ikonomou C, Williams B, Ramel S, Rolfe B, Kurol J. Orthodontic treatment standards in a public group practice in Sweden. Swed Dent J 2001;25:137-44. 21. Firestone AR, Beck M, Beglin FM, Vig KWL. Validity of the index of complexity, outcome, and need. Angle Orthod 2002;72: 15-20. 22. Richmond S, Ikonomou C, Williams B, Rolfe B. Orthodontic treatment standards in Greece. Hel Orthod Rev 2001;4:9-20. 23. Liepa A, Urtane I, Richmond S, Dunstan F. Orthodontic treatment need in Latvia. Eur J Orthod 2000;25:279-84. 24. Baume LJ, Marechaux S. Uniform methods for the epidemiologic assessment of malocclusion. Am J Orthod 1974;66:121-9. 25. McLain JB, Proffit WR. Oral health status in the United States: prevalence of malocclusion. J Dent Educ 1985;49:386-96. 26. Koochek A-R, Yeh MS, Rolfe B, Richmond S. The relationship between index of complexity, outcome and need, and patients’ perceptions of malocclusion: a study in general dental practice. Br Dent J 2001;191:325-9. 27. Roberts CT, Richmond S. The design and analysis of reliability studies for the use of epidemiological and audit indices in orthodontics. Br J Orthod 1997;24:139-47. 28. Fox NA, Daniels C, Gilgrass T. A comparison of the index of complexity, outcome and need (ICON) with the peer assessment rating (PAR) and the index of orthodontic treatment need (IOTN). Br Dent J 2002;193:225-30.