Sensitivity and Specificity of Lump Detection in Breast Models

Sensitivity and Specificity of Lump Detection in Breast Models Dana N. Rutledge, RN, PhD

Lump detection accuracy in models provides an index of ability to detect lumps in breast tissue. Recent research suggests poor lump detection accuracy in nontrained persons. Following individualized nurse-provided instruction in breast self-examination (BSE), 113 well-educated white women exhibited 89% mean lump detection sensitivity and 81 % mean specificity (sensitivity = number of lumps correctly identified divided by actual percentage of exams completed without lumps; specificity detection of false lumps). Problems with hands were potentially

linked to poor specificity. Completeness of search was significantly associated with sensitivity. Lumps most unlikely to be detected were those in the fleshy upper outer quadrant of torso models. High accuracy levels here may be attributable to timing of assessments, homogeneous sample, models used, numbers/ sizes of lumps, and long search times. BSE instruction should emphasize complete search, examination of outer quadrant of breasts, and assessment of hand problems. [Am J Prev Med 1992;8:314-8]

Before the mid-1980s, the emphasis in breast self-examination (BSE) research was on assessing self-reported frequency and proficiency and ensuring compliance. However, relationships between BSE frequency and measures of proficiency have been inconsistent. 1- 4 Another index of breast examination proficiency, lump detection accuracy (LDA) on silicone breast models, is available. Accuracy encompasses two related concepts: sensitivity and specificity.5 Sensitivity, the ability of a screening test to detect pathology, represents accurate number and position of lumps detected. Specificity enables correct interpretation of the absence of pathology. It is reflected in no falsepositive detections. Clinically, sensitivity is the more important factor. Lack of sensitivity means inability to detect pathology, which may affect morbidity and mortality from breast cancer. LDA in breast models has been associated with LDA in natural human breasts with known lesions. 6 Using silicone models, Fletcher et al. 7 found that women taught BSE using the Mammacare (Mammacare Corporation, Gainesville, Florida) method went from baseline sensitivities (percentage of lumps detected correctly) of 41 % to one-year postinstruction sensitivities of 57% (after pretest values con-

trolled for, P < .001). In the three groups of women in this study, pre- and posttest specificities (percentage of examinations without false lump detections) ranged from 63% to 73%. In a study aimed at assessing improvements in LDA among physicians and nurses using the Mammacare method, 8 results indicate that mean sensitivities increased from 57% to four-month sensitivities of 63% in the experimental group. Specificity decreased from 5 6% to 41 % in the experimental group but increased significantly in the control group. In a posttest-only control group design, Assaf et al. 2 assessed LDA among women taught BSE. Women who were taught individually by nurses using breast models and who received pamphlets attained sensitivities of 72 % three months following teaching. This figure contrasts with significantly lower sensitivities of women taught by video/pamphlet (48%) and pamphlet only (42%). In breast cancer patients who had done BSE prior to diagnosis, 3 82% of women found 60% or fewer lumps in a model. Nurses9 and physicians 10 were not much more proficient in detecting lumps. Over half of nurses found fewer than 50% of lumps in models,9 whereas physicians had a mean sensitivity of 44%. 10 Examination techniques found to relate to enhanced LDA are high number of correct steps used in breast exams 2·7; longer exam time3·6- 9 •11 ·12 • use of pads of the fingers 13 •14 • greater ' ~· ' . 9 •13 ; and use of the vertical strip amount of manual pressure method.7,8,15 Individual training in BSE with guided practice also relates to better accuracy.2,6,l6 This article describes a secondary analysis of data collected

=

From the Center for Nursing Research, Vanderbilt University School of Nursing, Vanderbilt University Medical Center, Nashville, Tennessee. Dr. Rutledge is now a consultant in nursing research, Irvine, California. Address reprint requests to Dr. Rutledge, 34 Deerwood East, Irvine, CA 92714-3025.

314 American Journal of Preventive Medicine, volume 8, number 5

for a study of effects of age on LDA. 1 7 Its aims are (1) to describe LDA in women who received a research-based teaching technique, and (2 ) to discuss associations between examination techniques and preexisting conditions with regard to LDA.

METHODS

Sample. Over a six-month period, women were recruited for BSE training from a variety of women's groups in two southeastern cities. Selection criteria were literacy, normal sight, and normal use of at least one arm. Procedures. During individual sessions with each woman, nurses explained procedures and the consent process. Women then filled out a demographic form, which included information about BSE history, breast health, and presence/absence of several neurological conditions potentially affecting motor/sensory abilities. Following The Special Touch, a sevenminute American Cancer Society film describing methods and purposes of BSE, nurses answered questions about the film and BSE issues. Next, guided practice on lifelike tabletop torso breast models (Health Edco, Waco, Texas) occurred. Models had one breast containing five lumps of varying sizes and on~ breast without lumps. During practice sessions, nurses emphasized a consistent search pattern, use of pads of the fingertips, variable pressure, thoroughness, and unhurried pace. Women were encouraged to select one search method from those shown in the film : circular, spoke, vertical strip. Most practice sessions lasted less than an hour. Women were not rushed. Following completion of the practice session, participants were assigned in random order to breast models for examination. Women examined two tabletop torso models, each containing two different breast lump configurations (see Figure 1). Immediately following each exam, women drew any lumps detected on a graphic outline of the model. Breast models and use. The two torso models were mirror images. According to random assignments, women were instructed to examine two specific breasts using a technique similar to what they would use on their own breasts. Three combinations of breast lump configurations were possible: (1) a breast with five lumps and one with three lumps, (2) two identical breasts containing three lumps, and (3) two identical

0

E

G Q

0

\ 0

Figure 1 L · ump configurations.

Oc

D

Bo

~

breasts containing five lumps. Thus, total number of lumps detectable ranged from six to 10. To simulate hand position for BSE, the torso models were examined upright on a table in front of participants. This method improved upon that in a pilot study, in which women palpated lumps in prone single-breast models on a table by pushing down. As in other studies, 7 silicone models tended to become "fatigued" over time. For this reason, practice models were isolated from exam models. Two complete sets of torso models were used for assessments. Data collection. I trained two registered nurses in BSE instruction methods, use of the breast models, and data collection methods. Each nurse scored participants on LDA immediately after a session. Nurses used a graphic key to the lump configuration outline to compare women's marks with actual lumps present. LDA was assessed by (1) sensitivity (percentage of lumps correctly detected in examinations of two breast models), and (2) specificity (percentage of examinations completed without identification of false lumps). Conferences including the cocollector of data, nurses, and me answered questions about accuracy scores; these questions occurred on two individual exams. Data collectors coded women's written breast lump assessments for model (five- or three-lump), number of correct and false lumps, length of exam (measured by stopwatch from time fingers first contacted the model until their final removal), and use of noted techniques. Techniques measured were use of pads of fingers (yes /no ), use of variable pressure (yes/no), and palpation of entire mammary area of the model (yes /no). Pattern of examination was categorized as circular, spoke, vertical strip, inconsistent across exams, or no pattern noted. Total BSE skill was not assessed since visual inspection was not included. Data analysis. Chi-square analyses assessed associations between sensitivity/specificity and categorical variables. Pearson product moment correlations assessed associations among interval level variables. The level of significance was .05. Findings Participants. One hundred thirteen women participated. Ages ranged from 22 to 86 years (mean [M] = 52, standard deviation [SD] = 16.3 ), and total years of education varied from 12 to 22 (M = 15, SD = 2.3; mode = 16). One woman was Asian; others were Caucasian. Most women (77%) had not experienced breast symptoms in the previous year. Four women (3.5% ) had received treatment for breast cancer, and 22 (19%) had a family history of breast cancer. Although 86 % of women had practiced BSE at some time, only 70% remembered having been taught BSE, most commonly by a pamphlet, a physician, or a tape (see Table 1). Mean number of times BSE had been practiced in the previous year was 6.6 (SD = 6.2; median = 6; mode = 12). Seven women (6%) had problems with the nerves in their hands; 10 (9%) had numbness/discomfort in their fingertips (only five of the seven women were in this 10). Two women had diabetes, and 21 (19%) had arthritis that affected hand joints. LDA. Mean lump detection sensitivity of women was 89%, with a range of 50%- 100% . Mean specificity was 81 %. Ninety-one women (81 % ) women found no false lumps in either exam, 18 (15% ) found at least one false lump in one exam, and four (4% ) found false lumps in both exams.

Am J Prev Med 1992;8(5) 315

Table 1. Types of BSE instruction

Table 2. Detection according to lump location

Question

% women responding yes

Have you ever been taught BSE? By physician? By nurse? By a taped message? In a group? Through written pamphlet? Read article in women's journal? Other? Have you ever done BSE?

79 39 16 31 12 43 27 3 86

I assessed problems that might contribute to poor ability to detect lumps accurately in association with LDA. Because of the small number of women with diabetes, I did no association analysis. For the other three conditions I assessed, no association existed with sensitivity. However, of the four women who detected false lumps in both exams, all indicated at least one problem with their hands, and one woman had all four. In women who detected at least one false lump, three (11 % ) had problems with nerves in their hands, four (16%) had fingertip numbness, one (5.3%) had diabetes, and four (16%) had arthritis in the hands. Since only 14% of women had not done BSE during the year prior to the study, no association existed between reported frequency of BSE and sensitivity/specificity. I did assess associations between LDA and type of BSE instruction. Women who had no instruction prior to this study did as well on sensitivity/specificity as instructed women. Associations between methods of previous instruction and sensitivity were negative, except that women who had received pamphlet instruction had higher levels of sensitivity than women who had not (P = .037). All four women who detected false lumps in both breasts had prior BSE instruction, as had 90% of all women who detected at least one false lump. Model examination patterns varied in the group. Most women (71 %) used the circular pattern, 4% used the spoke method, 20% used the vertical strip, and the rest (5%) used an inconsistent or indistinguishable pattern. There was no association between pattern and sensitivity or specificity. The only BSE technique having an association with sensitivity and specificity was the thoroughness of the search. Use of a complete search in both exams was significantly associated with sensitivity (P = .002). Only 18% of women who did not complete a thorough search attained 100% sensitivity, compared with 60% of women who did. Five (23%) of the women who found at least one false lump failed to do a complete search. Almost all participants used the pads of the fingers and variable pressure in their examinations, leading to inadequate cell sizes for chi-square analyses. Search time averaged 155 seconds (SD = 64) per exam. There was no correlation between search time and sensitivity or specificity. In the first five- and first three-lump assessment per woman, I evaluated placement and size of lump for ease of detection. Figure 1 shows the lump configurations with lumps labeled A-H. Table 2 indicates percentages of women who found each lump.

Lump

Description

% of women who detected

A B

2 cm; under outer nipple 1 cm; inner lower quadrant 1 cm; midline above nipple 0.5 cm; inner upper quadrant 0.3 cm; upper outer quadrant 1. 7 cm; under inner nipple 1 cm; upper outer quadrant 0.4 cm; midline below nipple

89.6 87.5 91.7 89.6 76.0 97.8 82.2 93.3

c

D E F G H

According to percentages of women not detecting the lumps, lumps in the upper outer quadrant are most difficult to detect (E,G). Those more easily detected are in the inner lower quad rant (F,H) on a model with fewer lumps and in both inner or lower quadrants (A,B,C,D) in a model with more lumps.

DISCUSSION The mean lump detection sensitivity of 89% attained in this sample of women is high. In the literature I surveyed, the closest sensitivity I found was the 72% attained at a threemonth follow-up 2 •1 6 in a group of women seeking routine cancer screening who were taught by nurses with breast models. Teaching sessions averaged 15 minutes. Other reports of sensitivities attained from studies with different designs 7 •8 •18 do show significant increases in sensitivity following teaching interventions. Worden 1 8 attained 44%-56% sensitivities in two groups of randomly selected women (only 40% of whom indicated receiving BSE training) at a point three years following initial intervention. Higher sensitivities occurred in women taught a simplified palpation technique in groups and followed with maintenance interventions. Fletcher et al. 7 attained 45%57% sensitivities across three groups one year following intervention. Women taught by nurses with the Mammacare method and followed by maintenance interventions had higher sensitivities than women taught by more traditional methods. Physicians and nurses who began with overall mean sensitivity of 57% 8 and received further breast examination instruction attained four-month follow-up sensitivity of 63 % (P :s .05). Other researchers have discussed strengths and weaknesses of using lump detection in silicone breasts as a measure of proficiency. 3·9·19 The biggest strength is its use as the standard method for judging performance of a skill. In actuality, smooth and regular models bear little resemblance to human breast tissue, especially to that of older women. Thus, results may be biased as an overestimate of a woman's ability to find lesions in her own breasts; on the other hand, they may be biased as understatements because of the nonreal "feel" to the models. Model examination certainly lacks the element of proporioception in a self-examination. 1 Several factors may contributl'to the high sensitivities attained in this study. Most salient are timing of the assessment and the study sample. Sensitivities described here were achieved immediately after an instructional session, when the chances for behavior change were highest. Most other studies described

316 American journal of Preventive Medicine, volume 8, number 5

here used longitudinal designs and thus LDA measurements further in time from the actual teaching session. Second, the selfselected and very well-educated sample optimized the teaching i~tervention . Most studies had quite different samples in terms of age and educational background. Other factors that may or may not contribute to the dissimilar findings are the model types used, the numbers and sizes of lumps in the models, and variability in search times for exams. Worden, 18 Fletcher, 7 • 10·12 and Campbell 8 used models constructed by Mammacare. Assaf, 2 Haughey, 3·9 and this study used models constructed by Health Edco. Perhaps lumps are harder to detect in Mammacare models than in Health Edco models. Numbers and sizes of lumps varied across studies. Worden 18 used two models containing 10 lumps; Fletcher 7 •10, 12 used six models containing 18 lumps, the largest being 1.0 cm; Assaf2,16 used two models with four lumps, 0.5-4 cm; this study used two models with six to 10 lumps, 0.3-2.0 cm. Average search times differed among participants across studies. Mean search time (155 seconds) in this study exceeds that found in other studies that cite searchtime, 3·9·15 except for the recent Campbell study.8 The lack of a pretest of women's LDAs is the major weakness of this study. Several recent studies offer some indication of lump detection sensitivities in untrained women/men. Only 21 % of nurses (M = 34 years) found all 10 lumps in two breast models 9; 54% found half or fewer. Only two breast cancer survivors3 found all five lumps in a breast model ; 75 % found 0-1 lumps. Kenney et al. 19 found mean sensitivity of 57% (SD = 28 ) in 73 untrained university women with a mean age of 27 years (lump sizes .4-.9 cm). Initial mean sensitivity of Worden's control groups was 29 % . Initial mean sensitivity of women in the study by Fletcher et al. 7 was 40 % (range 0%72% ). Studies of physician accuracy 8·10·12 indicate sensitivities of 44% (range 17%-83% ) in 80 randomly selected physicians, including 10 house staff physicians, which increases to 57%58 % in more recently trained residents. Considering the older mean age of this sample, mean sensitivities preinstruction probably were lower than the highest found in the reviewed studies (57%- 5 8 % ).7 ,8,l9 Thus, the potential increase in sensitivity from less than 58 % to 89% appears substantial. Eighty-one percent of the sample identified no false lumps. The mean specificity of 81 % was higher than that found in the literature. 7 ,s, 10,1 2 A difference in measurement of specificity in this study is a calculation of percentage based on only two examinations. This study suggests an association between specificity and problems with nerves in the hands and fingertip numbness. This finding is new and warrants further research. However, the fact that one-fifth of women detecting false lumps failed to use a complete search supports an association between thoroughness and specificity. 12 Factors associated with good LDA in this study are somewhat consistent with previous work. Completeness of search was the major predictor of sensitivity. Although increased search time was not a predictor, longer search times found in this study may explain this anomaly. Use of the vertical strip method was not a predictor. In a study of highly trained young women, Saunders et al.15 found vertical strip use associated with lncreased thoroughness of search compared with circular and spoke .methods; LDA was not measured. Perhaps in a more reahst1c setting with women receiving less training, complete~~t of search is more important than search technique used. s P0 mt needs further testing.

Lump size and location were factors in women's ability to detect. Others3· 8·9·10·12·16 have found that the larger the lump, the greater the chance of detection. This study indicates a possible interaction between lump size and lump location within breast tissue, with detection less likely for lumps in the fleshier portions of the breast. This point also warrants further study. Implications Instructors of BSE need to continue to emphasize completeness of search, variable pressure, use of fingertips, and adequate search time. Thorough examination of the fleshier upper outer quadrant of the breast where a preponderance of malignancies occur is essential. 20 Research into usefulness of the BSE instruction method described in this study needs to continue, especially with samples of diverse women and with a longitudinal design. Future studies should incorporate a pretest of women's LDA. Comparison of different breast models used for LDA measurements is needed. Also, another question needs study: are different ways of assessing specificity equivalent in terms of reliability and validity? Research that "unconfounds" instruction of the vertical strip method with other variables needs to be done. Further investigation into the potential association between hand problems and poor specificity is warranted. And, finally, a series of studies designed to test ways to better teach BSE would be useful. 21

This research was supported in part by grants from Inventive Products, Inc., Decatur, Illinois; the Dean's Development Fund, Vanderbilt Universiry School of Nursing; and the Vanderbilt Universiry Research Council. I thank Sharon Hendricks and Ursula Heitz for their help in data collection.

REFERENCES 1. Alagna S, Reddy D. Predictors of proficient technique and successful lesion detection in breast self-examination. Health Psycho! 1984;3: 11327. 2. Assaf, A, Cummings K, Walsh D. The relationship between breast self-examination frequency, technique and breast lump detection. In: Mettlin C, Murphy GP, eds. Progress in cancer control IV: research in the cancer center. New York: Alan R. Liss, 1983:323-9. 3. Haughey BP, Marshall JR, Nemoto T, Kroldart K, Mettlin C, Swanson M. Breast self-examination: reported practices, proficiency, and stage of disease at diagnosis. Oncol Nursing Forum 1988 ;15:3159. 4. Hislop TG, Coldman AJ, Skippen DH. Breast self-examination: importance of technique in early diagnosis. Can Med Assn J 1984;131 : 1349-52. 5. O'Malley MS, Fletcher SW. Screening for breast cancer with breast self-examination: a critical review. JAMA 1987;257:2197-2203. 6. Hall DC, Adams CK, Stein GH, Stephenson HS, Goldstein MK, Pennypacker HS. Improving detection of human breast lesions following experimental training. Cancer 1980;46:408-14. 7. Fletcher SW, O'Malley MS, Earp JL, Morgan TM, Lin S, Degnan D. How best to teach women breast self-examination: a randomized controlled trial. Ann Intern Med 1990;112:772-9. 8. Campbell HS, Fletcher SW, Lin S, Pilgrim CA, Morgan TA.

Am J Prev Med 1992;8(5) 317

Improving physicians' and nurses' clinical breast examination: a randomized controlled trial. Am J Prev Med 1991;7:1-8.

16. Assaf AR, Cummings K, Graham S, Mettlin C, Marshall JR. Comparison of three methods of teaching women how to perform breast self-examination. Health Educ Q 1985 ;12:259- 72.

9. Haughey B, Marshall J, Mettlin C, Nemoto T, Kroldart K, Swanson M. Nurses's ability to detect nodules in silicone breast models. Oncol Nursing Forum 1984;11:37-42.

17. Rutledge ON. Effects of age on lump detection accuracy. Nursing Res (in press).

10. Fletcher SW, O 'Malley MS, Pilgrim CA, Gonzalez JJ. How do women compare with internal medicine residents in breast lump detection? J Gen Intern Med 1989;4:277-83.

18. Worden JK, Solomon LJ, Flynn BS, Costanza MC, Foster RS, Dorwaldt AL, et al. A community-wide program in breast selfexamination training and maintenance. Prev Med 1990;19:254-69.

11. Adams C, Hall D, Pennypacker HS, Goldstein M, Henck L, Madden M, et al. Lump detection in simulated human breasts. Perception and Psychophysics 1976;20:163-7.

19. Kenney E, Hovell MF, Newborn CR, Elder JP. Breast selfexarnination among college women: predictors for cancer control. Arn J Prev Med 1989;5 :27-33.

12. Fletcher SW, O'Malley MS, Bunce L. Physicians' abilities to detect lumps in silicone breast models. JAMA 1985 ;253:2224-8 . 13. Howe H. Proficiency in performing breast self-examination. Patient Counsel Health Educ 1980;2:151-3.

20. Reichert CM, Moshiri S, Austin RM, Norris HG. Gross and microscopic pathology of female breast: prognostic variables and current approaches to primary breast cancer diagnosis. In: Lippman ME, Lichter AS, Danforth OW, eds. Diagnosis and management of breast cancer. Philadelphia: W.B. Saunders, 1980:22-48.

14. Mamon JA, Zapka JG. Determining validity of measuring the quality of breast self-examination . Eva] Health Professions 1985;8:5568.

21. Chrvala CA, Iverson DC. Predictive models for frequency and proficiency of BSE proficiency. In: Advances in cancer control: innovations and research. New York: Alan R. Liss, 1989:159-73.

15. Saunders KJ, Pilgrim CA, Pennypacker HS. Increasing proficiency of search in breast self-examination. Cancer 1986;58:2531-7.

318 American Journal of Preventive Medicine, volume 8, number 5