- Email: [email protected]
Continuing education: does it affect the practice of dentistry?
P E R S P E C T IV E S
C o n tin u in g th e
p r a c tic e
e d u c a tio n :
d o e s
it a ffe c t
o f d e n tis tr y ?
Gerard C. Kress, PhD
f
\_*lon tin u in g education has becom e increasingly important to the dental profession. It has grown from the status of brief refresher courses after m ili tary service during World War II to an increasingly com plex netw ork of courses. In addition to the growth in numbers of students and courses, there has been a recent growth in expectations about what the courses can accom plish. Significant num bers of state dental societies currently require con tinuing education as a condition of membership. Some state legislatures even require it for relicen sure. Both p olicies are based on the assum ption that continuing education influences the quality of dental care. Indeed, som e think that continuing education is the most desirable approach to com pliance w ith requirements of the Professional Re v iew Standards Organizations. These high expecta tions have grown even though there is little direct evidence concerning the effects of continuing edu cation on the practice of dentistry. T his paper review s what is know n about dental continuing education and discusses som e of the is sues that need to be addressed in research that is needed to fill in the gaps in our know ledge. The re v iew in clu d es sections on background and pur poses of continuing education, surveys of needs and resources of con tin u in g dental ed u cation , evaluation of continuing education in the health professions, and evaluation of the quality of dental care.
C o n tin u in g e d u c a tio n : b a c k g r o u n d a n d p u rp o s e s Before World War II, little need was perceived by dentists for continuing education. Dentists w ho re turned from military service wanted to be informed of current technical developm ents in the field. A c cordingly, refresher courses were offered by a net work of groups including dental societies, study clu b s, and d en tal sch o o ls. T he rapid p ace of technological change in the field created demand for more courses and their numbers have gradually 448
■ J A D A , V o l. 9 9 , S e p te m b e r 1 9 7 9
grown to im pressive am ounts.1 For exam ple, in the 1975-1976 scholastic year, dental schools offered 2,708 courses that were attended by 78,400 partici pants.2 In 1976, 2,661 courses were offered and were attended by 77,861 participants.3 These fig ures were substantially higher than those of the 1974 ADA survey that reported that 42,588 enrollees attended 1,471 courses. The figure reported in 1974 represents 43% of all practicing dentists in the country; however, as som e dentists took more than one course, they were counted two or more tim es and the number of individuals who took courses was smaller. Before 1969, continuing education of dentists w as totally voluntary. T his picture has rapidly changed. S in ce 1969, both state dental societies and state boards via legislatures—beginning w ith M innesota— have im posed mandatory continuing education requirements for membership or licen sure. By 1977, 18 states had set requirements at either the society or licensure level. Thus, continu ing education is seen as a m echanism for maintain ing and im proving d en tists’ ability and, hence, the quality of dental care. M ilgrom4 discussed the role of continuing educa tion in the establishm ent of a national standard for dental care. He em phasized a change in dental malpractice law that overturned the so-called “lo cality rule” as dem anding a single standard that could be com m unicated to the profession by a net work of continuing education courses. He asserted that current course efforts had totally failed and that requirem ents w ou ld have to becom e more rigorous to achieve the goal of establishing a na tional standard of care. This potential for continu ing education was reiterated by Cahn5 who d e scribed the im p lic a tio n s of the PSRO law. He pointed out that this legislation created the poten tial for establishm ent of a continuous network of continuing education and peer review. Such a net work w ould establish quality standards that could be substantiated and m odified through local, state, and regional system s. These standards could then
P E R S P E C T IV E S
be integrated into a national network that could exert professional control by identifying needs of training and ensuring standards of quality. No such network has developed but efforts have b een m ade to d ev elo p system s for m on itorin g course enrollm ents. Born,6 in 1974, described the d evelop m en t of the ADA C ontinuing Education Registry and the M innesota-based Dental Informa tion Services Center/Continuing Education Registry Service. The former began in 1972; the latter was started in 1973. The ADA Registry currently in cludes 25,000 enrollees from 12 states, the Veterans Adm inistration, and the Army; its function is that of record-keeping (L. L. Schuhrke, personal com m u n ica tio n ). Inform ation sy stem s of th is sort w ou ld be essential to the developm ent and adm in istration of the kind of network Cahn recom m ends. W aldm an7 proposed a plan to ensure that various needs w ou ld be served by mandatory, continuing ed u ca tio n . He su g g ested that requ irem en ts be structured to ensure that courses w ou ld be taken in several different areas. Moreover, he recom m ended that a c h ie v e m e n t of e x p lic it stan d ard s o f a c com plishm ent be required for course credit. O pinions about continuing education have not been u nan im ou s. For exam ple, in an editorial, Butts8 strongly protested against mandatory con tinuing education for dentists; no doubt he was speaking for many practitioners w ho are personally opposed to the concept. Chambers and H am ilton9 also were skeptical about som e of the expectations for continuing education. They took the position that the questions— H ow can continuing education protect the public? How can continuing education be more accessible to dentists? How can changes in behavior resulting from continuing education be measured?— are all based on false prem ises. They contend that it is unlikely that continuing educa tion can protect the public, that access is not a major determinant of dentist participation, and that pretest and posttest scores show ing gains are in v a lid m easures of p rofession al behavior. T hey suggest that alternate questions are more likely to p rod uce u sefu l an sw ers— H ow can co n tin u in g education better serve the profession? H ow can continuing education be made to fit the diverse needs of the profession? H ow can tests be d evel oped to diagnose the educational needs of ind i vidual dentists? H ow can those establishing con tinuing education courses cooperate w ith the state boards of exam iners to develop tests of com pe tency? H ozid10 also criticized the system of continuing education although he argued that it has the poten
tial to achieve a great deal. He offered the notion of “professional half-life” and estim ated that, after five years, dentists w ith ou t further training are likely to be halfway to total obsolescence. He also argued that programs of continuing education are also often obsolete and these programs may contri bute to professional obsolescence. H ozid pointed out that, because many older dentists were never trained in som e areas, (detection of oral cancer and scientific bases for many clinical practices) the “re fresher course” approach is often inappropriate. He urged that continuing education respond more to the needs of the profession and public. Johansen,11 in a 1974 editorial, said that steps should be taken to ensure that dentists w ou ld v o l untarily enroll in continuing education courses. A l though he acknow ledged that the voluntary system of professional self-im provem ent in dentistry has erred, he p o in ted out that more atten dan ce in courses “ . . . may mean little or nothing in terms of professional com petence.” The ADA Council on Dental Education set forth a list of g u id e lin e s for co n tin u in g ed u ca tio n in 197412 in response to the im position of require ments for continuing education and the prolifera tion of courses. A lthough strongly in favor of pro fessional participation in continuing education, the Council proposed steps to ensure the quality of courses. Briefly paraphrased, they included the fol lowing: — Profit-m aking organ ization s sh ou ld be d is couraged from sponsoring courses. — S p e cific cou rses sh o u ld be based on n eed rather than on potential for profit. — Evaluation com m ittees sh ould be set up by constituent societies. — Sponsoring agencies should appoint directors of continuing education. — Schedules of courses should be coordinated to avoid conflicts and overlapping subject matter. — Sponsors should ensure that ethical and legal procedures are taught. — Sponsors sh ou ld regulate p u b licity for the course. — C urriculum s sh o u ld be based on a ssessed needs of the region; class size should be held to a s iz e that p erm its ad eq uate p a rticip a tio n ; and m ethods should be used that encourage participa tion. — Content of courses and effectiveness of instruc tors should be evaluated and attention should be given to the health care needs of the com m unity. — F aculty sh ou ld be selected on the basis of teaching ability, technical com petence, and ethical Kress : C O N TIN U IN G EDUC ATIO N ■ 449
PERSPECTIVE S
standards. In its Annual Report to the 1978 ADA House of Delegates, the Council on Dental Education included a proposal for evaluation of continuing dental education and for an approval program. 13 The proposal includes 13 standards that sponsors of continuing education are expected to adhere to and follow to obtain and retain approval status . Briefly paraphrased , they are: -Administ ration of programs must be appropriate to achieve goals. -Fiscal resources must be adequate. -Goals must be stated by sponsors that relate to public health care and professiona l needs. -Specific educational objectives are to be prepared for each course. -Programs should be planned on the basis of assessment of objective needs. -Clear explication should be provided for special course prerequisite s and for methods for determining individual preparedne ss . -Education al methods must be appropriate to achieve course objectives. -Facilitie s must be appropriat e to achieve course objectives. -All necessary steps must be taken to ensure protection of patients who may be involved. -Faculty must be qualified and available in adequate numbers. -Methods of evaluation should include student achievemen t, estimates of effectivenes s of course and instructor, and periodic review of overall goals. -Publicity for courses should be informative and not misleading and should describe the course and its prerequisite s. -Course records should be accurate , permanent, accessible to participants , and should not include certificates or diplomas.
Expressed need and resources for continuin g education Several writers and the ADA pointed out that continuing education courses should be planned in recognition of professiona l and public health needs. The expressed needs of dental practitioner s have been surveyed several times and the results have been consistent. Others have surveyed the available resources. A review of studies of dentist attitudes about continuing education until 1969 was reported by Darby. 14 He reported that a high percentage (90%) of dentists indicated interest in continuing education. The report disclosed interest in the following 450 • JADA, Vol. 99, September 1979
order, from highest to lowest: crown and bridge; prosthetics; periodontic s; administrat ion of practice; general practice; endodontics ; new techniques and materials; diagnosis and treatment of emergencies; pretreatmen t and posttreatme nt medication; operative dentistry; oral surgery; and managemen t of child patients. Most dentists preferred participation courses, but the most common format was the lecture. Areas in which dentists reported the least desire for courses were radiology, public health, and the manag ement of special problems. Ryan,15 in another review , also reports on patterns of preference. Generally, dentists preferred short, half- to 11/z-day courses on clinical topics similar to those listed in Darby 's review. The most pref_erred method was clinical participatio n; the least preferred was self-instruc tion or home study. Cafferetta and others 16 surveyed 600 dentists from New York State. Their preferences were consistent with previous surveys . Interestingl y, 50% of those sampled favored mandatory continuing education. Several resource surveys have been reported. Chambers and Hamilton 17 surveyed the financial and physical resources committed to continuing dental education by American dental schools. They found that less than one school in four has classroom or laboratory space dedicated to continuing education, that less than half the programs are self-suppor ting , and , on the average, that less than one clinical participatio n course is offered each month. They concluded that these resources are inadequate ·to meet the need for larger numbers of higher quality courses. Van Stewart and Strauch 18 surveyed 20 Eastern dental schools about topics and settings for continuing education course offerings. The frequencies were highest in clinical disciplines. The five most frequent were orthodontic s , periodontic s , oral surgery, prosthodon tics, and operative dentistry. Tied for last place were community dentistry, forensic dentistry , hypnosis, nutrition, oral medicine , and public health. With some exceptions, these frequencies are consistent with the preferences of the dentist reported in earlier surveys. No courses were offered by department s such as biochemist ry , histology , microbiolo gy , oral anatomy, and pathology; this suggested that only the clinical needs of dentists are being served by these courses. Bird 1 surveyed 466 instructors of continuing dental education courses listed in the Journal and scheduled between July and December of 1975 . He reported that 101 of the courses were cancelled ,
P E R S P E C T IV E S
that m ean length of a course was eight hours during one to \ xk days, and 94% were betw een a half-day and five days long. Successful com pletion was de term ined by attendance in 37% of the courses, by first-day registration in 29%, a subjective evalua tion in 20%, and by objective testing in only 14%. He reported that a high percentage of the courses were “n ew ,” that 63% of the courses m ade a profit, and that 53% of the courses reportedly included participation. Pretests were reported for 5% of these courses; posttests were reported for 8%. Bird re ported that m ost instructors favored attendance at conventions for mandated con tin uin g education cred it but th ey w ere again st g iv in g cred it for courses offered by dental supply com panies. These surveys show that continuing dental edu cation is heavily oriented toward clinical m ethod ology as taught in sm all doses of short courses. There is little if any logical structure of courses of fered by the surveyed organizations (primarily den tal schools) and there is little evaluation of out com e. The high rate of cancellation for lack of enrollees and the focus on topics of great interest to clin ician s suggest that continuing dental education is a buyer’s market that has not yet been greatly af fected by mandatory requirements. The system was severely criticized by A d elson 19 as he argued for w id er u se of h o sp ita l-b a sed cou rses. He, w ith H o zid ,10 M ilgrom ,4 and others, argues that the majority of continuing education courses in den tistry have little or no favorable im pact on the qual ity of care delivered by the dentists w ho attend them.
E v a lu a t io n o f c o n t in u in g e d u c a tio n i n th e h e a lt h p ro fe s s io n s A lthough formal evaluation of courses does not seem to have been a high priority of developers of dental education courses, som e studies have been conducted to appraise the im pact of continuing education on the delivery of health care. An excel lent review of the issues and studies w as prepared by D ixon20 in 1977. She identifies four levels of criteria for evaluating continuing education pro grams: perception/opinion data; know ledge gain I attitude change data; clinical process data; and pa tient im pact data. Each su cceeding level produces more valid information but is more expensive and difficult to achieve. She describes three of these four levels as a causal chain of events; changes occur at each level and are dependent on change at each preceding level. She reasons that, in order for continuing education to affect health care, it m ust
do so by changes in clinician behavior that are, in turn, dependent on changes in know ledge or at titudes, or both, gained from the educational exp e rience. In other w ords, gains in k n o w led g e or changes in attitude, or both, are necessary but not su ffic ie n t to im p rove the c lin ic ia n ’s p ractice. Further, D ixon suggests that the ultim ate validation of the causal chain is provided by detectable im provem ents in patient health. She argues for more and better studies at the levels of higher criteria (clinical process and patient impact) and suggests that, if appropriate change is show n at either, it can be co n clu d ed that change also occurred at the lo w er le v e ls of o p in io n and lea rn in g /a ttitu d e change. PERCEPTION/OPINION MEASUREMENT IN CON TINUING EDUCATION. Because they are easy and inexpensive, opinion scales are the m ost com m only used evaluation m ethod in continuing education courses. Scales of this sort were fairly com m on among the courses in dentistry surveyed by Bird.1 He called them “h app iness” scales because they typically ask attendees to indicate degree of satis faction w ith various features of the course. A l though they may be useful for the formative evalua tion of a course, that is, for suggesting exp licit changes needed for im provem ent, few investigators w ould rely on them as valid indicators of course achievem ent, or of effective change in practice be havior. Perhaps the most dramatic illustration of the difficulty was a study by Naftulin and others21 in w h ich an actor, posing as a professor, delivered a lecture that consisted of double-talk and circular argum ents but presented it in a hum orous and charismatic manner. Mental health and education professionals in the audience gave h igh ly favorable ratings on the evaluation questionnaire to the lec ture. More recent studies have replicated this effect (the “Doctor Fox Effect”) among college students. The relevance of these studies to continuing educa tion in clinical areas was underlined by tw o studies of m edical education (McGuire and others22 and W illiam s and others23) in w hich, although (written) su b jectiv e ev a lu a tio n w as favorable, ob jectiv e measures show ed that desired changes in the clin i cal p ractice of th e p ractition ers had n ot b een achieved. KNOWLEDGE/ATTITUDE DATA. Formal tests may be m andated to certify ach ievem en t in con tin u in g dental education courses. According to Bird’s find ings, that w ou ld produce a major change in more than 90% of the courses offered. Whereas effects of Kress : C O N TIN U IN G EDUC ATIO N ■ 451
PERSPECTIVE S
formal tests might be desirable, there is evidence that such measures fall short of proving that clinical behavior will be improved. Chambers and others 24 pointed out that low scores can be faked on pretests in order to maximize scores of apparent gains in knowledge . Attitudes can certainly be faked on tests taken before and after the courses. Apart from the possibility of faking, scores on tests of factual knowledge are not always predictive of future nontest behavior. Dental students can sometimes perform well on a test of theory but can fail to apply the principles to practice in the laboratory or clinic. Written tests that are good predictors of subsequent academic performanc e , that is, grades, are usually not good predictors of success in nonschool activities. Thus, although acquisition of new knowledge, skills , and changes in attitudes may be necessary for improved clinical practice, they are insufficient . Gains in knowledge do not guarantee better practice behavior. CLINICAL PROCESS MEASURES. Despite the added
expense and difficulty, several investigator s have evaluated continuing education by seeking data at the clinical process level. For example, Boeck and others 25 collected self-reports of changes in the practice behavior of physicians three months after a discussion- oriented course in the field of endocrinology. They reported difficulty in getting replies: 47% responded after a round of a mailed questionnai re and a follow-up call. However, 16 of 18 respondent s reported at least one change in practice behavior as a result of the course. Another instance in which continuing education was evaluated by self-reporte d clinical process data was a study by Wilhelm. 26 She collected precourse and postcourse data on ten key procedures of physical therapy for evaluating the condition of patients who have suffered cerebrovasc ular accidents. Data were collected two months before and six months after several types of workshop courses. Although the results did not clearly show efficacy of the courses, the study exemplifies collection of practice outcome data. Garfin 27 used self-reports of dentists to evaluate a course in dentistry in hospitals. The participants reported that they treated more patients in hospitals after the course; he concluded that the course succeeded in changing the practice behaviors in the intended way. Condin 2 8 reported on evaluation of continuing education courses for nurses in which the supervisors of the participatin g nurses reported on the consistency of various aspects of the nursing process. The data suggested that the 452 • JADA, Vol. 99, September 1979
courses improved nursing behavior. Chambers and others 24 conducted a study of clinical processes in dentistry that provided an interesting precaution on the validity of self-report measures. They noticed overall improveme nt in ten aspects of four-handed dentistry during observations in the office made five months after a continuing education course. However, they also noticed a low rate of agreement between clinical observers and the dentists being observed with respect to evaluation of behavioral changes resulting from the course. The dentists not only modified what was presented in the course to fit their own specific needs and constraints, but they also focused on different changes in behavior in the responses to the questionnai re than did the evaluators. Another example of the use of clinical process measures was reported by McClellan and Cox 29 in 1968. They analyzed performanc e of dentists for the Indian Health Service by reviewing records of services performed before and after a course designed to improve operating efficiency. This method was so precise and direct· that they were able to calculate a numerical cost-benefit appraisal of the course for 18 months. The course was said to provide a sixfold return of costs because it added to productivit y during the 18 months. In 1977, Bird 1 tried a similar appraisal of continuing education in the Indian Health Service by using the records from the dental clinics maintained by the organization . He evaluated 39 dentists enrolled in six courses. Although he found a general increase in services delivered after the courses, the measures were not sufficiently sensitive to relate them to specific courses or to assure that the differences were really improveme nts. He conceded that changes in clinical technique, use of materials , and quality of service were not measured. Thus , the source of data that was valid for measuring the outcome of productivit y was less useful for other measures of quality of care. Several researchers in the field of medical education have used audits of records or observation of practice to evaluate continuing education. Most, however, have stopped short of examining patient outcomes. Kane and Bailey 30 counted the numbers of Pap smears done by physicians before and after a course. McGuire and others 22 investigated instruction in the evaluation of unknown sounds of the heart by a review of charts. Williamson and others 3 1 evaluated a workshop designed to increase response of physicians to unexpected abnormaliti es by examining records of patients . All three studies failed to produce evidence of improved practice
P E R S P E C T IV E S
w ith these third-level measures although measures of satisfaction and know ledge suggested that the courses had achieved their objectives. PATIENT-IMPACT MEASURES. A few studies used measures of patients’ health to evaluate continuing education. Lewis and H assanein32 evaluated a tenyear series of courses organized by the University of Kansas by associating rates of attendance w ith m aternal death rates, prenatal death rates, and surgery rates among the 11 regions of the state. A l though the design of the study was unusual, it did not control for the possibility that physicians in a region could choose to study topics most problema tic to the region. Thus, no firm conclusions could be drawn about cause and effect. Inue and others33 also used measures of patients’ health, in this case, to evaluate physician tutorials concerning treatment of hypertension. Am ong the measures collected six m onths after training were: tim e spent on physical exam ination, on history tak ing, and on patient education, and topics of chart notation. B esides these clinical behaviors, they also measured patients’ know ledge, com pliance, and blood pressure readings. The results suggested that the tutorials were h igh ly effective as measured both by im proved practices of physicians and im proved k n ow ledge and com plyin g behaviors of the pa tients. W einstein and others34 reported a significant cor relation betw een dentists’ expressed perceptions of the usefulness of university courses, study clubs and “talking shop” and the quality of treatment as measured by exam inations of patients. However, perceived effect of continuing dental education on quality and actual quality were not correlated. No relationship was found b etw een the num ber of courses taken and measured quality of treatment. A lthough variability in either numbers of courses or ratings of quality may have been lim ited and hence not conducive to producing high correla tions, the sam ple was large enough (1,196 dentists in the state of W ashington) to detect a relationship. H ozid and others35 reported a similar lack of corre lation betw een continuing education activity and dental “up-to-dateness.” No other know n study in continuing dental edu cation has used patient-impact measures. Only a few have used observation of clinical behavior and outcom e. Apart from the expense of these proce dures, it has been difficult to secure cooperation from practicing clinicians. Moreover, the effects of continuing dental education on the quality of den tal care is a current concern. Most system s of dental quality assurance have been developed recently.
E v a lu a t in g th e q u a lit y o f d e n t a l c a r e An excellent recent review of measures of dental quality by Cohen and Jago36 com prehensively pre sents the array that have been proposed. A focus of these measures has been on the status of oral health and oral disease of persons; another focus has been on broader issues of structure and functions of de livery system s as they relate to the status of oral health and oral disease in populations. Two of the best know n system s for evaluating single dental practices are Friedm an’s37 guide and the system described by Bailit and others38 in 1974. Both sys tem s inclu d e ratings of sp ecific procedures and their integration into a treatment plan. Each system evaluates com pleteness of exam ination and proper sequencing of treatment. Many other individual and group measures have been suggested and are review ed by Cohen and Jago.36 A com prehensive survey of the current state of the art in dental quality assurance m ethodology has re cently been published by the Am erican Dental A s sociation.39 This tw o-volum e report, Q uality assur ance in dentistry, includes a detailed review of 11 prom inent quality assurance system s, a bibliogra phy, a d iscussion of the major issu es and problems, recom m endations for future application of quality review system s, and needed research and d evelop ment. From the 1978 ADA study, a number of con clu sions can be m ade that have im plications for design of studies of quality of care delivered by solo practi tioners in noninstitutional settings. First, none of the system s review ed was suited to the private solo practitioners; all were geared toward institutional groups. Second, no system or approach w as w ithout serious conceptual or practical w eaknesses. It was concluded that the influence of continuing educa tion on quality be studied further to extend the find ing of W einstein and others34 that quality of care was related to the dentists’ perceived u sefu lness of u n i versity courses and study clubs. Three basic approaches to quality assessm ent are identified and compared: structural, process, and outcom e analyses. Structural criteria refer to aspects of the organization and resources that support the provision of dental care. A lthough this approach to quality assurance is appealing because it can be rela tively objective and unobtrusive, Bailit38 reported finding neither m uch variability in structural vari ables w h en review ing solo practice, nor any factors that influenced quality of care. Process criteria de scribe the procedures or performed therapies and m anagem ent of the disease. Every prom inent system included process criteria w ith u se of data from rec Kress : C O N TIN U IN G EDUC ATIO N ■ 453
PERSPECTIVE S
ords: claim forms, encounter forms, and patient records, or all three. Much less expensive and obtrusive than outcome criteria based on examination of patients , this approach demands adequate recording of information and reliable and valid measures of quality. Too often, records are inconsisten t, illegible, or simply incomplete. Thus, process measures can be difficult to collect from private practices. Outcome criteria are the most likely to be valid and usually include evaluation of treatment by patient examination . This procedure is relatively expensive, suffers from biased samples of patients who show up for examination , and is potentially deleterious to the relationship of dentist to patient. An approach to examining the process of care that appears to have much merit is the tracer methodology developed by Kessner and others 40 in medicine. Based on review of care provided for a single diagnostic category or surgical procedure, it uses explicit criteria. Tracers listed by Kessner and others included: otitis media and associated loss of hearing, visual disorders, iron deficiency anemia, urinary tract infection, essential hypertensio n, and cancer of the cervix. Each category or procedure identified as a tracer should meet these requirements: -A tracer should have a definite impact. -It should be well defined and easy to diagnose. -Rates of prevalence should be high enough to ensure an adequate sample of data. -Natural history of the condition should vary with use and effectivenes s of medical care. -Managem ent techniques should be well defined for at least one of these processes: prevention, diagnosis, treatment, or rehabilitatio n. -Effects of nonmedica l factors on the tracer · should be well understood. It is said that tracers can enhance objectivity of measureme nt so much that nonphysici ans could perform most of the review. As illustrations of the way in which tracers could be used to evaluate quality of routine care provided by a health care system, Kessner and others offer the following: "a center may be shown to care for only 11 % of the estimated population of adult males in the community ," and "drug therapy is found to meet minimal criteria in 30% of the treated patient sample." The tracer methodolog y seems a promising one for dentistry in which many procedures conform to Kessner's requiremen ts. Moreover, in studies by Brook41 (in medicine) and Bailit 38 (in dentistry), little variability was found in quality in a given practice. For example, Bailit reported quality of all procedures in his research to be highly correlated ex454 • JADA, Vol. 99, September 1979
cept for periodontal procedures that seemed to be independen t of the others . Thus , it may be possible to focus on a single procedure, perhaps the most common, and use it as the tracer that measures overall quality of dental service. Assuming that one procedure were , in fact , a valid measure of overall quality and that it could be evaluated by a record review rather than an examination of the patient, it would provide an advantagen ous method.
Where do we go from here? Need for research To formulate the most useful guidelines for continuing dental education, we need to know more about the connections between continuing dental education and the practicing behavior of dentists. This should include both those who enroll in courses and those who do not. Patterns of attendance in continuing dental education courses among the profession need to be documente d and correlated with patterns of practice. Current measures of practice structure, clinical process , and patients' outcome need to be identified and validated for the solo and small group practitioner s who represent the cross section of dental care in this country. Before we can compare a variety of continuing education experiences with each other, we must have measures that go beyond changes in knowledge and attitudes to actual behaviors of practice and their effects on patients. These measures should be objective, relatively convenient to collect, and should describe outcomes that are significant for the comfort and health of patients. It will not be easy to develop the measures and establish the relationsh ips. However, the alternative -to formulate policy on continuing dental education on the basis of intuition-i s not satisfying. Probably most people would agree that the effort required to produce the needed research will be justified by the assistance the results can provide to policy makers. If the profession is going to impose requiremen ts on its members , then it owes them a factual basis for these requiremen ts.
Summary Continuing education is becoming a requiremen t for membership in dental societies and, in some states, for dental licensure. These policies are based on the assumption that continuing education influences the quality of dental care. Critics of continuing dental education have suggested that this assumption may be unwarrante d. Because relatively
P E R S P E C T IV E S
little em phasis has been placed on formal evalua tion of continuing dental education courses in the past, the effects of these courses are not known. A review of the literature on evaluation of con tinuing education in the health professions show s that several different levels of evaluation criteria have been used. They in clu d e students’ percep tions and opinions about courses, gains in know l edge or changes in attitude, changes in clinical practice, and changes in patient’s satisfaction or health. The latter tw o levels, although more dif ficult to measure, are more clearly valid indicators of effectiveness of courses. Few studies on continu ing dental education have used either clinical pro cess or measures of im pact on patients. Research is needed to establish the connections betw een particular experiences in continuing den tal education and im proved quality of care. A major problem for such research is the lack of a clear defi nition and convenient measures of the quality of dental care. There is, therefore, a need to develop objective measures of change in practice behaviors that significantly affect the comfort and health of patients. W ithout these measures and the appropri ate research, gu id elin es for continuing education w ill lack a sound factual basis.
Dr. Kress is director of educational research, Harvard School of Dental Medicine, 188 Longwood Ave, Boston, 02115. Address requests for re prints to Dr. Kress. 1. Bird, W. A study of continuing dental education. Thesis, Harvard University, 1976. 2. Continuing education programs in dental schools. Annual Report, 1976-77, on Dental Education (suppl 6). 3. Continuing education in dental schools, 1977-78. Annual Report on Dental Education (suppl 6). 4. Milgrom, P. Continuing education and the prospect for a national standard of dental care. JDent Educ 38:482-486, 1974. 5. Cahn, E.A. Professional standards review organizations: a new con cept in dental continuing education. JColo Dent Assoc 53(1):15-16,1974. 6. Born, D.O. An information systems approachto monitoring continu ing dental education activity. JDent Educ 38:491-493,1974. 7. Waldman, H.B. A plan for structural standards for mandatory con tinuing education. NY State Dent J38:302-304, 1972. 8. Butts, H.C. Mandatory continuing education conflicts with true pro fessionalism. JADA91(2):197-198, 1975. 9. Chambers, D.W., and Hamilton, D.L. Continuing dental education: reasonable answers tounreasonablequestions. JADA90(1):116-120,1975. 10. Hozid, J.L. Role of continuing education in dental obsolescence. JADA79(6):1299-1303, 1969. 11. Johansen, E. Professional self-improvement. J Dent E d u c 38:481, 1974. 12. CDE proposes continuing education guidelines. J Dent Educ 38:521-522, 1974. 13. American Dental Association Council on Dental Education. Stan dards for approval of institutions and organizations offering continuing dental education programs. Appendix III-B, Annual Report to the 1978 ADA House of Delegates, 1978. 14. Darby, D. The dentist and continuing education—attitudes and motivations. JAmer Coll Dent 36:165-170,1969.
15. Ryan, R.M. Continuing education research. In Richards, N.D., and Cohen, L.K. (eds.). Social sciences and dentistry. A critical bibliography. The Hague, A. Sythoff, 1971. 16. Cafferetta, G.L., and others. Continuing education: attitudes, inter ests, and experiences of practicing dentists. JDent Educ 39(12):793-800, 1975. 17. Chambers, D.W., and Hamilton, D.L. A survey of resources of con tinuing education programs in American dental schools. J Dent Educ 39(4):234-238, 1975. 18. Van Stewart, A., and Strauch, A.B. Study of continuing education programs in selected Easterndental schools. JDent Educ41(2):86-88,1977. 19. Adelson, R. The role of the hospital in continuing dental education. JDent Educ 38:487-490, 1974. 20. Dixon, J.K. Methodological considerations in evaluating continuing education in the health professions—utilization of chain of events models in determining evaluation criteria. Presented at the Annual Convention of the American Educational Research Association, New York City, April, 1977. 21. Naftulin, D.H.; Ware, J.E.; and Donnelly, F.A. The Doctor Fox lec ture: a paradigm of educational seduction. JMed Educ 48:630-635,1973. 22. McGuire, C., and others. Auscultatory skill: gain and retention after extensive instruction. JMed Educ 39:120-131,1964. 23. Williams, R.G., and Ware, J.E. Validity of student ratings of instruc tion under different incentive conditions: afurther study of the Doctor Fox effect. Transactions of the convention of the AmericanEducation Research Association. Washington, DC, 1975. 24. Chambers, D.W., and others. An investigation of behavior change in continuing dental education. JDent Educ 40(8):546-551, 1976. 25. Boeck, M.A.; Wood, F.C.; and Dohner, C.W. Continuing education: assessment of value of discussion sessions and of course impact on physi cian practice. Northwest Med 71:519-522,1972. 26. Wilhelm, I.J. Evaluation of behavioral change reported by partici pants in two types of postgraduate education programs. Phys Ther 52:751-757, 1972. 27. Garfin, L.A. An evaluation of hospital dental seminars. Northwest Dent 48:167-168, 1969. 28. Condon, S.B.M. Inservice education . . . impact on patient care. J Cont Educ Nursing 5:44-51, 1974. 29. McClellan, T.E., and Cox, J.L. Description and evaluation of dentist-dental assistant teamtraining in efficient dental practice manage ment. JADA 76(3):548-553, 1968. 30. Kane, R., and Bailey, R.Q. Evaluation of a postgraduate educational programme in early cancer detection. Br JMed Educ 5:134-137,1971. 31. Williamson, J.W.; Alexander, M.; and Miller, G.E. Continuing edu cation and patient care research: physician response to screening test re sults. JAMA 201:118-122, 1967. 32. Lewis, C.E., and Hassanein, R.S. Continuing medical education—an epidemiological evaluation. New Eng JMed 282:254-259,1970. 33. Inue, T.S.; Yourtee, E.L.; and Williamson, J.W. Improved outcomes in hypertension after physician tutorials: a controlled trial. Ann Intern Med 84(6):646-651, 1976. 34. Weinstein, P., and others. Quality and perceived usefulness and utilization of continuing education. Presented at the 55th Annual Session IADRCopenhagen, April, 1977. IADRAbstracts, no. 134,1977. 35. Hozid, J.; Clark, R.E.; and Kornman, J.H. The measurement of upto-dateness in dentistry. Presented at the 50th General Session, IADR, Las Vegas, March, 1972. IADRAbstracts, no. 103,1972. 36. Cohen, L., and Jago, J. Toward the formulation of sociodental indi cators. Int JHealth Serv 6:681-688, 1976. 37. Friedman, J.W. A guide for the evaluation of dental care. Los Angeles, School of Public Health, University of California, 1972. 38. Bailit, H., and others. Quality of dental care: development of stan dards. JADA89(4):842-853, 1974. 39. Stern, S.K.; Morrissey, S.C.; and Mauldin, J.E. Quality assurance in dentistry, vol I: The state of the art of dental quality assurance. Chicago, American Dental Association, 1978. 40. Kessener, D.M.; Kalk, C.E.; and Singer, J. Assuring health quality— the case for tracers. New Eng JMed 288:189-194,1973. 41. Brook, R.H.; Williams, K.N.; and Avery, A.D. Quality assurance todayandtomorrow: forecast forthe future. Ann InternMed85:809,1976. Kress : C O N TIN U IN G EDUC ATIO N ■ 455
C o n tin u in g th e
p r a c tic e
e d u c a tio n :
d o e s
it a ffe c t
o f d e n tis tr y ?
Gerard C. Kress, PhD
f
\_*lon tin u in g education has becom e increasingly important to the dental profession. It has grown from the status of brief refresher courses after m ili tary service during World War II to an increasingly com plex netw ork of courses. In addition to the growth in numbers of students and courses, there has been a recent growth in expectations about what the courses can accom plish. Significant num bers of state dental societies currently require con tinuing education as a condition of membership. Some state legislatures even require it for relicen sure. Both p olicies are based on the assum ption that continuing education influences the quality of dental care. Indeed, som e think that continuing education is the most desirable approach to com pliance w ith requirements of the Professional Re v iew Standards Organizations. These high expecta tions have grown even though there is little direct evidence concerning the effects of continuing edu cation on the practice of dentistry. T his paper review s what is know n about dental continuing education and discusses som e of the is sues that need to be addressed in research that is needed to fill in the gaps in our know ledge. The re v iew in clu d es sections on background and pur poses of continuing education, surveys of needs and resources of con tin u in g dental ed u cation , evaluation of continuing education in the health professions, and evaluation of the quality of dental care.
C o n tin u in g e d u c a tio n : b a c k g r o u n d a n d p u rp o s e s Before World War II, little need was perceived by dentists for continuing education. Dentists w ho re turned from military service wanted to be informed of current technical developm ents in the field. A c cordingly, refresher courses were offered by a net work of groups including dental societies, study clu b s, and d en tal sch o o ls. T he rapid p ace of technological change in the field created demand for more courses and their numbers have gradually 448
■ J A D A , V o l. 9 9 , S e p te m b e r 1 9 7 9
grown to im pressive am ounts.1 For exam ple, in the 1975-1976 scholastic year, dental schools offered 2,708 courses that were attended by 78,400 partici pants.2 In 1976, 2,661 courses were offered and were attended by 77,861 participants.3 These fig ures were substantially higher than those of the 1974 ADA survey that reported that 42,588 enrollees attended 1,471 courses. The figure reported in 1974 represents 43% of all practicing dentists in the country; however, as som e dentists took more than one course, they were counted two or more tim es and the number of individuals who took courses was smaller. Before 1969, continuing education of dentists w as totally voluntary. T his picture has rapidly changed. S in ce 1969, both state dental societies and state boards via legislatures—beginning w ith M innesota— have im posed mandatory continuing education requirements for membership or licen sure. By 1977, 18 states had set requirements at either the society or licensure level. Thus, continu ing education is seen as a m echanism for maintain ing and im proving d en tists’ ability and, hence, the quality of dental care. M ilgrom4 discussed the role of continuing educa tion in the establishm ent of a national standard for dental care. He em phasized a change in dental malpractice law that overturned the so-called “lo cality rule” as dem anding a single standard that could be com m unicated to the profession by a net work of continuing education courses. He asserted that current course efforts had totally failed and that requirem ents w ou ld have to becom e more rigorous to achieve the goal of establishing a na tional standard of care. This potential for continu ing education was reiterated by Cahn5 who d e scribed the im p lic a tio n s of the PSRO law. He pointed out that this legislation created the poten tial for establishm ent of a continuous network of continuing education and peer review. Such a net work w ould establish quality standards that could be substantiated and m odified through local, state, and regional system s. These standards could then
P E R S P E C T IV E S
be integrated into a national network that could exert professional control by identifying needs of training and ensuring standards of quality. No such network has developed but efforts have b een m ade to d ev elo p system s for m on itorin g course enrollm ents. Born,6 in 1974, described the d evelop m en t of the ADA C ontinuing Education Registry and the M innesota-based Dental Informa tion Services Center/Continuing Education Registry Service. The former began in 1972; the latter was started in 1973. The ADA Registry currently in cludes 25,000 enrollees from 12 states, the Veterans Adm inistration, and the Army; its function is that of record-keeping (L. L. Schuhrke, personal com m u n ica tio n ). Inform ation sy stem s of th is sort w ou ld be essential to the developm ent and adm in istration of the kind of network Cahn recom m ends. W aldm an7 proposed a plan to ensure that various needs w ou ld be served by mandatory, continuing ed u ca tio n . He su g g ested that requ irem en ts be structured to ensure that courses w ou ld be taken in several different areas. Moreover, he recom m ended that a c h ie v e m e n t of e x p lic it stan d ard s o f a c com plishm ent be required for course credit. O pinions about continuing education have not been u nan im ou s. For exam ple, in an editorial, Butts8 strongly protested against mandatory con tinuing education for dentists; no doubt he was speaking for many practitioners w ho are personally opposed to the concept. Chambers and H am ilton9 also were skeptical about som e of the expectations for continuing education. They took the position that the questions— H ow can continuing education protect the public? How can continuing education be more accessible to dentists? How can changes in behavior resulting from continuing education be measured?— are all based on false prem ises. They contend that it is unlikely that continuing educa tion can protect the public, that access is not a major determinant of dentist participation, and that pretest and posttest scores show ing gains are in v a lid m easures of p rofession al behavior. T hey suggest that alternate questions are more likely to p rod uce u sefu l an sw ers— H ow can co n tin u in g education better serve the profession? H ow can continuing education be made to fit the diverse needs of the profession? H ow can tests be d evel oped to diagnose the educational needs of ind i vidual dentists? H ow can those establishing con tinuing education courses cooperate w ith the state boards of exam iners to develop tests of com pe tency? H ozid10 also criticized the system of continuing education although he argued that it has the poten
tial to achieve a great deal. He offered the notion of “professional half-life” and estim ated that, after five years, dentists w ith ou t further training are likely to be halfway to total obsolescence. He also argued that programs of continuing education are also often obsolete and these programs may contri bute to professional obsolescence. H ozid pointed out that, because many older dentists were never trained in som e areas, (detection of oral cancer and scientific bases for many clinical practices) the “re fresher course” approach is often inappropriate. He urged that continuing education respond more to the needs of the profession and public. Johansen,11 in a 1974 editorial, said that steps should be taken to ensure that dentists w ou ld v o l untarily enroll in continuing education courses. A l though he acknow ledged that the voluntary system of professional self-im provem ent in dentistry has erred, he p o in ted out that more atten dan ce in courses “ . . . may mean little or nothing in terms of professional com petence.” The ADA Council on Dental Education set forth a list of g u id e lin e s for co n tin u in g ed u ca tio n in 197412 in response to the im position of require ments for continuing education and the prolifera tion of courses. A lthough strongly in favor of pro fessional participation in continuing education, the Council proposed steps to ensure the quality of courses. Briefly paraphrased, they included the fol lowing: — Profit-m aking organ ization s sh ou ld be d is couraged from sponsoring courses. — S p e cific cou rses sh o u ld be based on n eed rather than on potential for profit. — Evaluation com m ittees sh ould be set up by constituent societies. — Sponsoring agencies should appoint directors of continuing education. — Schedules of courses should be coordinated to avoid conflicts and overlapping subject matter. — Sponsors should ensure that ethical and legal procedures are taught. — Sponsors sh ou ld regulate p u b licity for the course. — C urriculum s sh o u ld be based on a ssessed needs of the region; class size should be held to a s iz e that p erm its ad eq uate p a rticip a tio n ; and m ethods should be used that encourage participa tion. — Content of courses and effectiveness of instruc tors should be evaluated and attention should be given to the health care needs of the com m unity. — F aculty sh ou ld be selected on the basis of teaching ability, technical com petence, and ethical Kress : C O N TIN U IN G EDUC ATIO N ■ 449
PERSPECTIVE S
standards. In its Annual Report to the 1978 ADA House of Delegates, the Council on Dental Education included a proposal for evaluation of continuing dental education and for an approval program. 13 The proposal includes 13 standards that sponsors of continuing education are expected to adhere to and follow to obtain and retain approval status . Briefly paraphrased , they are: -Administ ration of programs must be appropriate to achieve goals. -Fiscal resources must be adequate. -Goals must be stated by sponsors that relate to public health care and professiona l needs. -Specific educational objectives are to be prepared for each course. -Programs should be planned on the basis of assessment of objective needs. -Clear explication should be provided for special course prerequisite s and for methods for determining individual preparedne ss . -Education al methods must be appropriate to achieve course objectives. -Facilitie s must be appropriat e to achieve course objectives. -All necessary steps must be taken to ensure protection of patients who may be involved. -Faculty must be qualified and available in adequate numbers. -Methods of evaluation should include student achievemen t, estimates of effectivenes s of course and instructor, and periodic review of overall goals. -Publicity for courses should be informative and not misleading and should describe the course and its prerequisite s. -Course records should be accurate , permanent, accessible to participants , and should not include certificates or diplomas.
Expressed need and resources for continuin g education Several writers and the ADA pointed out that continuing education courses should be planned in recognition of professiona l and public health needs. The expressed needs of dental practitioner s have been surveyed several times and the results have been consistent. Others have surveyed the available resources. A review of studies of dentist attitudes about continuing education until 1969 was reported by Darby. 14 He reported that a high percentage (90%) of dentists indicated interest in continuing education. The report disclosed interest in the following 450 • JADA, Vol. 99, September 1979
order, from highest to lowest: crown and bridge; prosthetics; periodontic s; administrat ion of practice; general practice; endodontics ; new techniques and materials; diagnosis and treatment of emergencies; pretreatmen t and posttreatme nt medication; operative dentistry; oral surgery; and managemen t of child patients. Most dentists preferred participation courses, but the most common format was the lecture. Areas in which dentists reported the least desire for courses were radiology, public health, and the manag ement of special problems. Ryan,15 in another review , also reports on patterns of preference. Generally, dentists preferred short, half- to 11/z-day courses on clinical topics similar to those listed in Darby 's review. The most pref_erred method was clinical participatio n; the least preferred was self-instruc tion or home study. Cafferetta and others 16 surveyed 600 dentists from New York State. Their preferences were consistent with previous surveys . Interestingl y, 50% of those sampled favored mandatory continuing education. Several resource surveys have been reported. Chambers and Hamilton 17 surveyed the financial and physical resources committed to continuing dental education by American dental schools. They found that less than one school in four has classroom or laboratory space dedicated to continuing education, that less than half the programs are self-suppor ting , and , on the average, that less than one clinical participatio n course is offered each month. They concluded that these resources are inadequate ·to meet the need for larger numbers of higher quality courses. Van Stewart and Strauch 18 surveyed 20 Eastern dental schools about topics and settings for continuing education course offerings. The frequencies were highest in clinical disciplines. The five most frequent were orthodontic s , periodontic s , oral surgery, prosthodon tics, and operative dentistry. Tied for last place were community dentistry, forensic dentistry , hypnosis, nutrition, oral medicine , and public health. With some exceptions, these frequencies are consistent with the preferences of the dentist reported in earlier surveys. No courses were offered by department s such as biochemist ry , histology , microbiolo gy , oral anatomy, and pathology; this suggested that only the clinical needs of dentists are being served by these courses. Bird 1 surveyed 466 instructors of continuing dental education courses listed in the Journal and scheduled between July and December of 1975 . He reported that 101 of the courses were cancelled ,
P E R S P E C T IV E S
that m ean length of a course was eight hours during one to \ xk days, and 94% were betw een a half-day and five days long. Successful com pletion was de term ined by attendance in 37% of the courses, by first-day registration in 29%, a subjective evalua tion in 20%, and by objective testing in only 14%. He reported that a high percentage of the courses were “n ew ,” that 63% of the courses m ade a profit, and that 53% of the courses reportedly included participation. Pretests were reported for 5% of these courses; posttests were reported for 8%. Bird re ported that m ost instructors favored attendance at conventions for mandated con tin uin g education cred it but th ey w ere again st g iv in g cred it for courses offered by dental supply com panies. These surveys show that continuing dental edu cation is heavily oriented toward clinical m ethod ology as taught in sm all doses of short courses. There is little if any logical structure of courses of fered by the surveyed organizations (primarily den tal schools) and there is little evaluation of out com e. The high rate of cancellation for lack of enrollees and the focus on topics of great interest to clin ician s suggest that continuing dental education is a buyer’s market that has not yet been greatly af fected by mandatory requirements. The system was severely criticized by A d elson 19 as he argued for w id er u se of h o sp ita l-b a sed cou rses. He, w ith H o zid ,10 M ilgrom ,4 and others, argues that the majority of continuing education courses in den tistry have little or no favorable im pact on the qual ity of care delivered by the dentists w ho attend them.
E v a lu a t io n o f c o n t in u in g e d u c a tio n i n th e h e a lt h p ro fe s s io n s A lthough formal evaluation of courses does not seem to have been a high priority of developers of dental education courses, som e studies have been conducted to appraise the im pact of continuing education on the delivery of health care. An excel lent review of the issues and studies w as prepared by D ixon20 in 1977. She identifies four levels of criteria for evaluating continuing education pro grams: perception/opinion data; know ledge gain I attitude change data; clinical process data; and pa tient im pact data. Each su cceeding level produces more valid information but is more expensive and difficult to achieve. She describes three of these four levels as a causal chain of events; changes occur at each level and are dependent on change at each preceding level. She reasons that, in order for continuing education to affect health care, it m ust
do so by changes in clinician behavior that are, in turn, dependent on changes in know ledge or at titudes, or both, gained from the educational exp e rience. In other w ords, gains in k n o w led g e or changes in attitude, or both, are necessary but not su ffic ie n t to im p rove the c lin ic ia n ’s p ractice. Further, D ixon suggests that the ultim ate validation of the causal chain is provided by detectable im provem ents in patient health. She argues for more and better studies at the levels of higher criteria (clinical process and patient impact) and suggests that, if appropriate change is show n at either, it can be co n clu d ed that change also occurred at the lo w er le v e ls of o p in io n and lea rn in g /a ttitu d e change. PERCEPTION/OPINION MEASUREMENT IN CON TINUING EDUCATION. Because they are easy and inexpensive, opinion scales are the m ost com m only used evaluation m ethod in continuing education courses. Scales of this sort were fairly com m on among the courses in dentistry surveyed by Bird.1 He called them “h app iness” scales because they typically ask attendees to indicate degree of satis faction w ith various features of the course. A l though they may be useful for the formative evalua tion of a course, that is, for suggesting exp licit changes needed for im provem ent, few investigators w ould rely on them as valid indicators of course achievem ent, or of effective change in practice be havior. Perhaps the most dramatic illustration of the difficulty was a study by Naftulin and others21 in w h ich an actor, posing as a professor, delivered a lecture that consisted of double-talk and circular argum ents but presented it in a hum orous and charismatic manner. Mental health and education professionals in the audience gave h igh ly favorable ratings on the evaluation questionnaire to the lec ture. More recent studies have replicated this effect (the “Doctor Fox Effect”) among college students. The relevance of these studies to continuing educa tion in clinical areas was underlined by tw o studies of m edical education (McGuire and others22 and W illiam s and others23) in w hich, although (written) su b jectiv e ev a lu a tio n w as favorable, ob jectiv e measures show ed that desired changes in the clin i cal p ractice of th e p ractition ers had n ot b een achieved. KNOWLEDGE/ATTITUDE DATA. Formal tests may be m andated to certify ach ievem en t in con tin u in g dental education courses. According to Bird’s find ings, that w ou ld produce a major change in more than 90% of the courses offered. Whereas effects of Kress : C O N TIN U IN G EDUC ATIO N ■ 451
PERSPECTIVE S
formal tests might be desirable, there is evidence that such measures fall short of proving that clinical behavior will be improved. Chambers and others 24 pointed out that low scores can be faked on pretests in order to maximize scores of apparent gains in knowledge . Attitudes can certainly be faked on tests taken before and after the courses. Apart from the possibility of faking, scores on tests of factual knowledge are not always predictive of future nontest behavior. Dental students can sometimes perform well on a test of theory but can fail to apply the principles to practice in the laboratory or clinic. Written tests that are good predictors of subsequent academic performanc e , that is, grades, are usually not good predictors of success in nonschool activities. Thus, although acquisition of new knowledge, skills , and changes in attitudes may be necessary for improved clinical practice, they are insufficient . Gains in knowledge do not guarantee better practice behavior. CLINICAL PROCESS MEASURES. Despite the added
expense and difficulty, several investigator s have evaluated continuing education by seeking data at the clinical process level. For example, Boeck and others 25 collected self-reports of changes in the practice behavior of physicians three months after a discussion- oriented course in the field of endocrinology. They reported difficulty in getting replies: 47% responded after a round of a mailed questionnai re and a follow-up call. However, 16 of 18 respondent s reported at least one change in practice behavior as a result of the course. Another instance in which continuing education was evaluated by self-reporte d clinical process data was a study by Wilhelm. 26 She collected precourse and postcourse data on ten key procedures of physical therapy for evaluating the condition of patients who have suffered cerebrovasc ular accidents. Data were collected two months before and six months after several types of workshop courses. Although the results did not clearly show efficacy of the courses, the study exemplifies collection of practice outcome data. Garfin 27 used self-reports of dentists to evaluate a course in dentistry in hospitals. The participants reported that they treated more patients in hospitals after the course; he concluded that the course succeeded in changing the practice behaviors in the intended way. Condin 2 8 reported on evaluation of continuing education courses for nurses in which the supervisors of the participatin g nurses reported on the consistency of various aspects of the nursing process. The data suggested that the 452 • JADA, Vol. 99, September 1979
courses improved nursing behavior. Chambers and others 24 conducted a study of clinical processes in dentistry that provided an interesting precaution on the validity of self-report measures. They noticed overall improveme nt in ten aspects of four-handed dentistry during observations in the office made five months after a continuing education course. However, they also noticed a low rate of agreement between clinical observers and the dentists being observed with respect to evaluation of behavioral changes resulting from the course. The dentists not only modified what was presented in the course to fit their own specific needs and constraints, but they also focused on different changes in behavior in the responses to the questionnai re than did the evaluators. Another example of the use of clinical process measures was reported by McClellan and Cox 29 in 1968. They analyzed performanc e of dentists for the Indian Health Service by reviewing records of services performed before and after a course designed to improve operating efficiency. This method was so precise and direct· that they were able to calculate a numerical cost-benefit appraisal of the course for 18 months. The course was said to provide a sixfold return of costs because it added to productivit y during the 18 months. In 1977, Bird 1 tried a similar appraisal of continuing education in the Indian Health Service by using the records from the dental clinics maintained by the organization . He evaluated 39 dentists enrolled in six courses. Although he found a general increase in services delivered after the courses, the measures were not sufficiently sensitive to relate them to specific courses or to assure that the differences were really improveme nts. He conceded that changes in clinical technique, use of materials , and quality of service were not measured. Thus , the source of data that was valid for measuring the outcome of productivit y was less useful for other measures of quality of care. Several researchers in the field of medical education have used audits of records or observation of practice to evaluate continuing education. Most, however, have stopped short of examining patient outcomes. Kane and Bailey 30 counted the numbers of Pap smears done by physicians before and after a course. McGuire and others 22 investigated instruction in the evaluation of unknown sounds of the heart by a review of charts. Williamson and others 3 1 evaluated a workshop designed to increase response of physicians to unexpected abnormaliti es by examining records of patients . All three studies failed to produce evidence of improved practice
P E R S P E C T IV E S
w ith these third-level measures although measures of satisfaction and know ledge suggested that the courses had achieved their objectives. PATIENT-IMPACT MEASURES. A few studies used measures of patients’ health to evaluate continuing education. Lewis and H assanein32 evaluated a tenyear series of courses organized by the University of Kansas by associating rates of attendance w ith m aternal death rates, prenatal death rates, and surgery rates among the 11 regions of the state. A l though the design of the study was unusual, it did not control for the possibility that physicians in a region could choose to study topics most problema tic to the region. Thus, no firm conclusions could be drawn about cause and effect. Inue and others33 also used measures of patients’ health, in this case, to evaluate physician tutorials concerning treatment of hypertension. Am ong the measures collected six m onths after training were: tim e spent on physical exam ination, on history tak ing, and on patient education, and topics of chart notation. B esides these clinical behaviors, they also measured patients’ know ledge, com pliance, and blood pressure readings. The results suggested that the tutorials were h igh ly effective as measured both by im proved practices of physicians and im proved k n ow ledge and com plyin g behaviors of the pa tients. W einstein and others34 reported a significant cor relation betw een dentists’ expressed perceptions of the usefulness of university courses, study clubs and “talking shop” and the quality of treatment as measured by exam inations of patients. However, perceived effect of continuing dental education on quality and actual quality were not correlated. No relationship was found b etw een the num ber of courses taken and measured quality of treatment. A lthough variability in either numbers of courses or ratings of quality may have been lim ited and hence not conducive to producing high correla tions, the sam ple was large enough (1,196 dentists in the state of W ashington) to detect a relationship. H ozid and others35 reported a similar lack of corre lation betw een continuing education activity and dental “up-to-dateness.” No other know n study in continuing dental edu cation has used patient-impact measures. Only a few have used observation of clinical behavior and outcom e. Apart from the expense of these proce dures, it has been difficult to secure cooperation from practicing clinicians. Moreover, the effects of continuing dental education on the quality of den tal care is a current concern. Most system s of dental quality assurance have been developed recently.
E v a lu a t in g th e q u a lit y o f d e n t a l c a r e An excellent recent review of measures of dental quality by Cohen and Jago36 com prehensively pre sents the array that have been proposed. A focus of these measures has been on the status of oral health and oral disease of persons; another focus has been on broader issues of structure and functions of de livery system s as they relate to the status of oral health and oral disease in populations. Two of the best know n system s for evaluating single dental practices are Friedm an’s37 guide and the system described by Bailit and others38 in 1974. Both sys tem s inclu d e ratings of sp ecific procedures and their integration into a treatment plan. Each system evaluates com pleteness of exam ination and proper sequencing of treatment. Many other individual and group measures have been suggested and are review ed by Cohen and Jago.36 A com prehensive survey of the current state of the art in dental quality assurance m ethodology has re cently been published by the Am erican Dental A s sociation.39 This tw o-volum e report, Q uality assur ance in dentistry, includes a detailed review of 11 prom inent quality assurance system s, a bibliogra phy, a d iscussion of the major issu es and problems, recom m endations for future application of quality review system s, and needed research and d evelop ment. From the 1978 ADA study, a number of con clu sions can be m ade that have im plications for design of studies of quality of care delivered by solo practi tioners in noninstitutional settings. First, none of the system s review ed was suited to the private solo practitioners; all were geared toward institutional groups. Second, no system or approach w as w ithout serious conceptual or practical w eaknesses. It was concluded that the influence of continuing educa tion on quality be studied further to extend the find ing of W einstein and others34 that quality of care was related to the dentists’ perceived u sefu lness of u n i versity courses and study clubs. Three basic approaches to quality assessm ent are identified and compared: structural, process, and outcom e analyses. Structural criteria refer to aspects of the organization and resources that support the provision of dental care. A lthough this approach to quality assurance is appealing because it can be rela tively objective and unobtrusive, Bailit38 reported finding neither m uch variability in structural vari ables w h en review ing solo practice, nor any factors that influenced quality of care. Process criteria de scribe the procedures or performed therapies and m anagem ent of the disease. Every prom inent system included process criteria w ith u se of data from rec Kress : C O N TIN U IN G EDUC ATIO N ■ 453
PERSPECTIVE S
ords: claim forms, encounter forms, and patient records, or all three. Much less expensive and obtrusive than outcome criteria based on examination of patients , this approach demands adequate recording of information and reliable and valid measures of quality. Too often, records are inconsisten t, illegible, or simply incomplete. Thus, process measures can be difficult to collect from private practices. Outcome criteria are the most likely to be valid and usually include evaluation of treatment by patient examination . This procedure is relatively expensive, suffers from biased samples of patients who show up for examination , and is potentially deleterious to the relationship of dentist to patient. An approach to examining the process of care that appears to have much merit is the tracer methodology developed by Kessner and others 40 in medicine. Based on review of care provided for a single diagnostic category or surgical procedure, it uses explicit criteria. Tracers listed by Kessner and others included: otitis media and associated loss of hearing, visual disorders, iron deficiency anemia, urinary tract infection, essential hypertensio n, and cancer of the cervix. Each category or procedure identified as a tracer should meet these requirements: -A tracer should have a definite impact. -It should be well defined and easy to diagnose. -Rates of prevalence should be high enough to ensure an adequate sample of data. -Natural history of the condition should vary with use and effectivenes s of medical care. -Managem ent techniques should be well defined for at least one of these processes: prevention, diagnosis, treatment, or rehabilitatio n. -Effects of nonmedica l factors on the tracer · should be well understood. It is said that tracers can enhance objectivity of measureme nt so much that nonphysici ans could perform most of the review. As illustrations of the way in which tracers could be used to evaluate quality of routine care provided by a health care system, Kessner and others offer the following: "a center may be shown to care for only 11 % of the estimated population of adult males in the community ," and "drug therapy is found to meet minimal criteria in 30% of the treated patient sample." The tracer methodolog y seems a promising one for dentistry in which many procedures conform to Kessner's requiremen ts. Moreover, in studies by Brook41 (in medicine) and Bailit 38 (in dentistry), little variability was found in quality in a given practice. For example, Bailit reported quality of all procedures in his research to be highly correlated ex454 • JADA, Vol. 99, September 1979
cept for periodontal procedures that seemed to be independen t of the others . Thus , it may be possible to focus on a single procedure, perhaps the most common, and use it as the tracer that measures overall quality of dental service. Assuming that one procedure were , in fact , a valid measure of overall quality and that it could be evaluated by a record review rather than an examination of the patient, it would provide an advantagen ous method.
Where do we go from here? Need for research To formulate the most useful guidelines for continuing dental education, we need to know more about the connections between continuing dental education and the practicing behavior of dentists. This should include both those who enroll in courses and those who do not. Patterns of attendance in continuing dental education courses among the profession need to be documente d and correlated with patterns of practice. Current measures of practice structure, clinical process , and patients' outcome need to be identified and validated for the solo and small group practitioner s who represent the cross section of dental care in this country. Before we can compare a variety of continuing education experiences with each other, we must have measures that go beyond changes in knowledge and attitudes to actual behaviors of practice and their effects on patients. These measures should be objective, relatively convenient to collect, and should describe outcomes that are significant for the comfort and health of patients. It will not be easy to develop the measures and establish the relationsh ips. However, the alternative -to formulate policy on continuing dental education on the basis of intuition-i s not satisfying. Probably most people would agree that the effort required to produce the needed research will be justified by the assistance the results can provide to policy makers. If the profession is going to impose requiremen ts on its members , then it owes them a factual basis for these requiremen ts.
Summary Continuing education is becoming a requiremen t for membership in dental societies and, in some states, for dental licensure. These policies are based on the assumption that continuing education influences the quality of dental care. Critics of continuing dental education have suggested that this assumption may be unwarrante d. Because relatively
P E R S P E C T IV E S
little em phasis has been placed on formal evalua tion of continuing dental education courses in the past, the effects of these courses are not known. A review of the literature on evaluation of con tinuing education in the health professions show s that several different levels of evaluation criteria have been used. They in clu d e students’ percep tions and opinions about courses, gains in know l edge or changes in attitude, changes in clinical practice, and changes in patient’s satisfaction or health. The latter tw o levels, although more dif ficult to measure, are more clearly valid indicators of effectiveness of courses. Few studies on continu ing dental education have used either clinical pro cess or measures of im pact on patients. Research is needed to establish the connections betw een particular experiences in continuing den tal education and im proved quality of care. A major problem for such research is the lack of a clear defi nition and convenient measures of the quality of dental care. There is, therefore, a need to develop objective measures of change in practice behaviors that significantly affect the comfort and health of patients. W ithout these measures and the appropri ate research, gu id elin es for continuing education w ill lack a sound factual basis.
Dr. Kress is director of educational research, Harvard School of Dental Medicine, 188 Longwood Ave, Boston, 02115. Address requests for re prints to Dr. Kress. 1. Bird, W. A study of continuing dental education. Thesis, Harvard University, 1976. 2. Continuing education programs in dental schools. Annual Report, 1976-77, on Dental Education (suppl 6). 3. Continuing education in dental schools, 1977-78. Annual Report on Dental Education (suppl 6). 4. Milgrom, P. Continuing education and the prospect for a national standard of dental care. JDent Educ 38:482-486, 1974. 5. Cahn, E.A. Professional standards review organizations: a new con cept in dental continuing education. JColo Dent Assoc 53(1):15-16,1974. 6. Born, D.O. An information systems approachto monitoring continu ing dental education activity. JDent Educ 38:491-493,1974. 7. Waldman, H.B. A plan for structural standards for mandatory con tinuing education. NY State Dent J38:302-304, 1972. 8. Butts, H.C. Mandatory continuing education conflicts with true pro fessionalism. JADA91(2):197-198, 1975. 9. Chambers, D.W., and Hamilton, D.L. Continuing dental education: reasonable answers tounreasonablequestions. JADA90(1):116-120,1975. 10. Hozid, J.L. Role of continuing education in dental obsolescence. JADA79(6):1299-1303, 1969. 11. Johansen, E. Professional self-improvement. J Dent E d u c 38:481, 1974. 12. CDE proposes continuing education guidelines. J Dent Educ 38:521-522, 1974. 13. American Dental Association Council on Dental Education. Stan dards for approval of institutions and organizations offering continuing dental education programs. Appendix III-B, Annual Report to the 1978 ADA House of Delegates, 1978. 14. Darby, D. The dentist and continuing education—attitudes and motivations. JAmer Coll Dent 36:165-170,1969.
15. Ryan, R.M. Continuing education research. In Richards, N.D., and Cohen, L.K. (eds.). Social sciences and dentistry. A critical bibliography. The Hague, A. Sythoff, 1971. 16. Cafferetta, G.L., and others. Continuing education: attitudes, inter ests, and experiences of practicing dentists. JDent Educ 39(12):793-800, 1975. 17. Chambers, D.W., and Hamilton, D.L. A survey of resources of con tinuing education programs in American dental schools. J Dent Educ 39(4):234-238, 1975. 18. Van Stewart, A., and Strauch, A.B. Study of continuing education programs in selected Easterndental schools. JDent Educ41(2):86-88,1977. 19. Adelson, R. The role of the hospital in continuing dental education. JDent Educ 38:487-490, 1974. 20. Dixon, J.K. Methodological considerations in evaluating continuing education in the health professions—utilization of chain of events models in determining evaluation criteria. Presented at the Annual Convention of the American Educational Research Association, New York City, April, 1977. 21. Naftulin, D.H.; Ware, J.E.; and Donnelly, F.A. The Doctor Fox lec ture: a paradigm of educational seduction. JMed Educ 48:630-635,1973. 22. McGuire, C., and others. Auscultatory skill: gain and retention after extensive instruction. JMed Educ 39:120-131,1964. 23. Williams, R.G., and Ware, J.E. Validity of student ratings of instruc tion under different incentive conditions: afurther study of the Doctor Fox effect. Transactions of the convention of the AmericanEducation Research Association. Washington, DC, 1975. 24. Chambers, D.W., and others. An investigation of behavior change in continuing dental education. JDent Educ 40(8):546-551, 1976. 25. Boeck, M.A.; Wood, F.C.; and Dohner, C.W. Continuing education: assessment of value of discussion sessions and of course impact on physi cian practice. Northwest Med 71:519-522,1972. 26. Wilhelm, I.J. Evaluation of behavioral change reported by partici pants in two types of postgraduate education programs. Phys Ther 52:751-757, 1972. 27. Garfin, L.A. An evaluation of hospital dental seminars. Northwest Dent 48:167-168, 1969. 28. Condon, S.B.M. Inservice education . . . impact on patient care. J Cont Educ Nursing 5:44-51, 1974. 29. McClellan, T.E., and Cox, J.L. Description and evaluation of dentist-dental assistant teamtraining in efficient dental practice manage ment. JADA 76(3):548-553, 1968. 30. Kane, R., and Bailey, R.Q. Evaluation of a postgraduate educational programme in early cancer detection. Br JMed Educ 5:134-137,1971. 31. Williamson, J.W.; Alexander, M.; and Miller, G.E. Continuing edu cation and patient care research: physician response to screening test re sults. JAMA 201:118-122, 1967. 32. Lewis, C.E., and Hassanein, R.S. Continuing medical education—an epidemiological evaluation. New Eng JMed 282:254-259,1970. 33. Inue, T.S.; Yourtee, E.L.; and Williamson, J.W. Improved outcomes in hypertension after physician tutorials: a controlled trial. Ann Intern Med 84(6):646-651, 1976. 34. Weinstein, P., and others. Quality and perceived usefulness and utilization of continuing education. Presented at the 55th Annual Session IADRCopenhagen, April, 1977. IADRAbstracts, no. 134,1977. 35. Hozid, J.; Clark, R.E.; and Kornman, J.H. The measurement of upto-dateness in dentistry. Presented at the 50th General Session, IADR, Las Vegas, March, 1972. IADRAbstracts, no. 103,1972. 36. Cohen, L., and Jago, J. Toward the formulation of sociodental indi cators. Int JHealth Serv 6:681-688, 1976. 37. Friedman, J.W. A guide for the evaluation of dental care. Los Angeles, School of Public Health, University of California, 1972. 38. Bailit, H., and others. Quality of dental care: development of stan dards. JADA89(4):842-853, 1974. 39. Stern, S.K.; Morrissey, S.C.; and Mauldin, J.E. Quality assurance in dentistry, vol I: The state of the art of dental quality assurance. Chicago, American Dental Association, 1978. 40. Kessener, D.M.; Kalk, C.E.; and Singer, J. Assuring health quality— the case for tracers. New Eng JMed 288:189-194,1973. 41. Brook, R.H.; Williams, K.N.; and Avery, A.D. Quality assurance todayandtomorrow: forecast forthe future. Ann InternMed85:809,1976. Kress : C O N TIN U IN G EDUC ATIO N ■ 455