- Email: [email protected]
Incremental prognostic power of clinical history, exercise electrocardiography and myocardial perfusion scintigraphy in suspected coronary artery disease
The incremental abilii of a clinical history, exercise electrocardiiraphy (ECG) and myocardial perfusion Scintiiraphy to identtfy coronary events in the year after testing was assessed in 1,659 patients with symptoms suggestive of coronary artery diiease (CAD), 74 of whom suffered a coronary event in the year after testing. Prognostic power was quantified in terms of the area under receiver operating characteristic curves derived from logistic regression. In 1,451 patii with normal rest ECG findings, a clinkal history alone provided the most prognostlc power (area = 72%). This improved signiiintly (by 5 % ) only when both tests were analyzed. In contrast, clinkal history had significant-
E
xercise electrocardiography (ECG) and myocardial perfusion scintigraphy are both widely used for prognostic assessment of patients with suspected coronary artery disease (CAD).*-I5 It is likely, however, that the practical value of these tests would be enhanced if they were used in a natural clinical hierarchy, whereby each test is used only if it provides information in addition to that provided by other tests lower in the From the Division of Cardiology and Department of Nuclear Medicine, Cedars-Sinai Medical Center, Los Angeles, California and the Department of Medicine, UCLA School of Medicine, Los Angeles, California. This study was funded in part by Specialized Center of Research (SCOR) Grant (HL-17651) and by Training Grant 2T32HL07380 from the National Heart, Lung, and Blood Institute, National Institutes of Heath, Bethesda, Maryland. Manuscript received August 14.1986; revised manuscript received September 15, 1986, accepted September 17, 1986. Address for reprints: George A. Diamond, MD, Division of Cardiology, Cedars-Sinai Medical Center, Post Office Box 48750, Los Angeles, California 90048.
ly less prognostk power in the 206 patients with abnormal rest ECG findii (area = 56%), but each test then provided a signifkant incremental improvement in these patients (by 14 % for each). A strategic model was thereby developed for prognostic assessment that recognizes the incremental power of these tests in specific patient groups as well as their overall accuracy and monetary cost. Thii strategy stratii individual patient risk for subsequent coronary events over a full order of magnitude (from 2 to 22%) at a 64% reduction in the cost of testing compared to performing both stress tests In all patients. (Am J Cardii 1967;59:270-277)
hierarchy. Thus, exercise ECG must add to the information provided by the clinical history alone, and perfusion scintigraphy must add to that provided by the combined analysis of both the clinical history and exercise ECG. Accordingly, this study was designed to evaluate the incremental prognostic power of clinical history, exercise ECG and myocardial perfusion scintigraphy to define a rational strategy for prognostic testing.
Methods Patients: Between January 1,1979, and September 30, 1982, a total of 3,155 patients underwent exercise redistribution perfusion scintigraphy at Cedars-Sinai Medical Center and entered a prospective follow-up program. For this study, we excluded 790 patients with a history of myocardial infarction, 464 patients who had already undergone coronary bypass surgery, 12 patients who died of various noncardiac causes and 30 patients in whom any of the variables needed for statistical analysis were missing. Two hundred additional patients were excluded because of incomplete follow-
February
TABLE
I
Clinical
Characteristics
Patient
Study Population (n = 1.659)
Characteristic Age W Sex Male Female Symptom class Asymptomatic Nonanginal Atypical Typical Stress electrocardiogaphy Achieved HFI (%) ST >I mm Perfusion scintigraphy savmax II savmax >l CAD = coronary
of Primary
I, 1987
artery
55f
12
26 14 24 56
11
56f
(21%) (12%) (20%) (48%)
20 20 18 20
up; 122 of these were referred for coronary bypass surgery soon after noninvasive testing (within 60 days) and 78 were lost due to geographic relocation. The final study group consisted of 1,659 patients without documented CAD. The patients lost to follow-up tended to be similar to study patients, whereas those referred for early surgery tended to be sicker than study patients (Table I). Clinical history: All patients were interviewed by a cardiology fellow before exercise testing. Clinical history was characterized in terms of age, sex and symptoms (classified with respect to substernal location, exertional precipitation and prompt relief by rest or nitroglycerin). Typical angina was defined by the presence of all 3 characteristics, atypical angina by any 2 and nonanginal discomfort by fewer than 2; patients who denied discomfort were termed asymptomatic.16 The probability of CAD was calculated from age, sex and symptom class using a computer program CADENZA) developed in our laboratory.” Electrocardiography: Supine and standing l&lead ECGs were recorded before exercise testing. Rest ECG findings were considered abnormal if they revealed bundle branch block (left or right) or ST-T-wave changes consistent with left ventricular hypertrophy or digitalis effect. Symptom-limited treadmill exercise testing was performed using the Bruce protocol. Betablocking agents were discontinued for 24 hours and nitrates for 4 hours before testing. The following exercise ECG variables were measured: peak magnitude of ST depression (80 ms after the J point), ST slope (upsloping, horizontal or downsloping). achieved level of stress (achieved heart rate divided by maximal predicted heart rate) and exercise-limiting chest pain. Perfusion scintigraphy: At peak exercise, patients received an injection of 2 mCi of thallium-201; exercise was continued for 60 seconds after injection. Scintigraphic imaging was begun 5 minutes after exercise (postexercise imaging) and repeated 3 to 6 hours after injection of thallium (redistribution imaging). All images (anterior, 45O and 70~ left oblique projections) were interpreted by 2 or more experienced observers
59
271
= maximal
scan
11
(26%) (26%) (23%) (26%)
91 f 14 41 (53%)
16 (13%) 106 (67%) rate; Sevmax
Volume
58 (74%) 20 (26%)
83 f 13 101 (63%)
1.197 (73%) 462 (27%)
OF CARDIOLOGY
Lost to Follow-up (n = 76)
102 (64%) 20 (16%)
(27%) (34%) (22%) (18%)
HR = hearl
JOURNAL
Groups
59*
95f 11 827 (50%)
disease:
AMERICAN
Early Sqery (n = 122)
1,122 (67%) 557 (33%) 443 557 365 292
THE
43 (55%) 35 (45%) severity.
without knowledge of the clinical history and exercise ECG results. Hypoperfusion was scored in each of 5 segments using a 4-point ordinal scale (0 = none, 1 = mild, 2 = moderate, 3 = marked).13 Only the maximal severity of hypoperfusion (the score with the greatest value at peak exercise) and the extent of reversible hypoperfusion (the number of hypoperfused segments which improve by a score of 1 or more with redistribution) were analyzed because these variables alone provide the most prognostic information contained in the scintigraphic images.13 Follow-up: Patients were contacted by telephone 1 year after testing. Follow-up data were obtained by means of a scripted interview administered by trained personnel. The following outcomes were considered a cardiac event: cardiac death (confirmed by review of the death certificate and hospital chart); nonfatal myocardial infarction (documented by a consistent history, ECG changes and cardiac enzyme elevations, and confirmed by hospital chart review); and referral for coronary artery bypass surgery more than 60 days after noninvasive testing (documented by hospital chart review)-a proxy for clinical deterioration16 that represents a compromise between including all referrals for bypass surgery as events (a liberal approach resulting in overestimation of the actual event rate) and excluding all patients referred for bypass surgery (a conservative approach resulting in underestimation of the actual event rate).13 Logistic analysis: Logistic regression analysis was used to estimate the probability of a cardiac event based on various combinations of input.lg For patients with normal rest ECG results, 4 models were evaluated: Model 1 used the preexercise probability of disease based on the historical variables (age. sex, symptom class); model 2 used all historical variables in addition to the preexercise probability of disease (on the assumption that this diagnostic probability might not distill all the prognostic information contained in the variables themselves): model 3 used the preexercise probability of disease, all the historical variables and all exercise ECG variables; and model 4 used the
272
INCREMENTAL
TABLE Groups
II
ACCUFtACY
Cllnlcal
OF CORONARY
Characteristics
ARTERY
DISEASE
PROGNOSIS
of Electrocardkgraphlc
Rest Electrocardiogram Characteristic
Ncrmal
55 f
Age (~0 sex Male Female Symptom class Asymptomatic Nonanginal Atypical Typical Stress electrocardiography Achieved l-R (%) ST >l mm Perhkon sclntigraphy sevmax I1 savmax >I Abbreviations
as in Table
(n = 1.451) 12
971 (87%) 480 (33%) 370 507 323 252
(28 %) (35%) (22%) (17%)
Abnormal 58f
(n = 208) 13
151 (73%) 57 (27 %) 73 52 42 41
(35%) (25%) (20%) (20%)
95f 11 723 (51%)
94 f 15 104 (50%)
1.076 (74%) 375 (26%)
121 (57%) 87 (42%)
I.
preexercise probability of disease, all historical variables, all exercise ECG variables and all scintigraphic variables. For patients with abnormal ECG findings, 3 models were evaluated: Model 1 used all historical variables alone; model 2 used all historical and exercise ECG variables; and model 3 used all historical, exercise ECG and scintigraphic variables. The coefficients for each model are available on request. Prognostic power: The receiver operating characteristic curve represents the relation between a test’s true-positive rate and false-positive rate as the threshold for abnormality is changed.20J1 The prognostic
power of each logistic model was quantified in terms of the area under this curve. For each logistic model, a continuous curve was constructed from the true- and false-positive rates for all possible thresholds of event probability.22.23 A difference in area defined the increment in prognostic power between different models. Statistical comparisons of these areas were performed using the method of Hanley and McNei1.24 When comparing proportional differences between groups, Fisher’s exact test was used. A p value <0.05 was chosen to indicate statistical significance.
Results Clinical outcome: The clinical characteristics of the study patients are summarized in Tables I and II. The total coronary event rate was 5% (76 events in 1,659 patients: 11 deaths, 24 nonfatal infarctions and 41 late surgical referrals]. The event rate was 4% (62 events in 1.451 patients) in those with normal ECG findings and 7% (14 events in 208 patients) in those with abnormal ECG findings (p = 0.16). Patients with normal rest electrocardiograms: Receiver operating characteristic curves for the 4 logistic models are shown in Figure 1 and the area under each curve is shown in Figure 2. Although all areas were of similar magnitude, they gradually increased with each sequential testing strategy. The increment in areas was 2 f 5% (72 - 70, p = 0.29). with analysis of individual historical variables 3 f 5% (75 - 72, p = 0.20) with exercise ECG and 2 f 5% (77 - 75, p = 0.28) with scintigraphy. The overall increment in area was 6 f 5% (77 - 71, p = 0.04). Thus, disease probability alone provided most of the prognostic power, while a small but significant improvement was provided by exercise ECG and scintigraphy. Prognostic subsets defined by disease probability: Because l-vessel disease is more likely when disease
80
0
0.5 FALSE POSITIVE
Ip=.04-----2
1.0 RATE
FIGURE 1. Receiver operatlng characterlstlc curves for patients with normal rest ekctrocardlographk flndings. From the lowest to the highest, these curves represent preexerclse disease probabgfty (P) alone, probabillty plus hlstory (Pi-H), and probabkty plus hlstory plus exercise electrocardiogram (ECG) (P-l-H+E), and probabillty plus history plus exercise ECG plus thallium sckttgraphy (P+H+E+T).
FIGURE 2. Receiver operating characterktk (RGC) curve areas for patlents wlth normal rest electrocardkgraphk findings. Each bar represents a mean value and associated standard devlatkn. Although individual Increments in curve area were small, the overall Increment (from P to P-i-H+E+T) was statistically sfgnifkant. An area of 50% is equivalent to an Information content of 0. Abbreviations as in Ffgure 1.
February
probability is less than 25% and multivessel disease is more likely when disease probability is greater than 75%,17 we chose these otherwise arbitrary thresholds to better define patients in whom noninvasive testing had its greatest incremental prognostic power. Low probability: The 746 patients with a preexercise disease probability of less than 25% represented the largest subpopulation (45% of total) and the population with the lowest event rate (16 events, 2%). In this subgroup, receiver operating characteristic curve area did not increase with exercise ECG (-7 f 10% [65 721, p = 0.75) or with scintigraphy (-4 f 11% [61- 651, P = 0.49) (Fig. 3). Noninvasive testing, therefore, provided no incremental prognostic information in patients with a low disease probability based on age, sex and symptoms. Intermediate probability: The smaller subpopulation of 494 patients (30% of total) with a disease probability between 25% and 75% had an event rate of 3% (17 events], twice that of the low-disease-probability group (p = 0.21). There was a trend toward increasing receiver operating characteristics curve area with both exercise ECG (6 f 10% [70 - 641, p = 0.16) and scintigraphy (8 f 10% [78 - 701, p = 0.14). This trend reached significance when the overall increment from clinical history alone to clinical history, exercise ECG and scintigraphy was analyzed (p = 0.01) (Fig. 3). Use of both tests, therefore, provided additional prognostic information in this subset. High probability: The 211 patients (13% of total] with a disease probability of more than 75% had an event rate of 14% (29 events), 4 times that of patients with an intermediate disease probability (p
1, 1967
THE AMERICAN
*1
r Ti T
OF CARDIOLOGY
I--***1
T EVENT RATE (%I
lo
r -
*p=.o1
* *p = ,005 **p=.oo1
EXERCISE ELECTROCARDIOGRAPHY PRE-EXERCISE DISEASE PROBABILITY
n=
FIGURE 3. Prognostic power in probability subsets. Patients are separated into 3 groups based on preexercise dlsease probability: low (p <25%), intermediate (25%
75 %). At low probability, neither exercise electrocardiography nor scintigraphy provided incremental improvement In prognostic power. At intermedlate probability, individual increments were still small, but the overall increment was statistically signlflcant. At high probability, the increment for exercise electrocardiography alone became significant. Abbreviations as in Figures 1 and 2.
Volume
59
273
0.005]. Perfusion scintigraphy again had a modest additional effect on receiver operating characteristic curve area, similar to that in the intermediate-diseaseprobability group (Fig. 3). The average increment in area was 5 f 7% (73 - 68, p = 0.14). Combined testing with exercise ECG and scintigraphy resulted in an overall incremental gain of 18 f 7% (p
20
80
JOURNAL
nn -
2
+
380
-
YTERMEDlATl
LOW 296
*1* ,,I -++
70
162
275
+
+
HIGH
57
55
99
57
FIGURE 4. Exercise electrocardiography and prognosis. Patients are grouped by disease probablllty (Fig. 3) and by the exercise electrocardiographic response. Exercise electrocardiographic results were consldered negative (-) (< 1 mm ST depression, I55 % maximal heart rate), discordant (+) (
274
INCREMENTAL
ACCURACY
OF CORONARY
ARTERY
DISEASE
PROGNOSIS
area averaged only 58 f 8%. Both noninvasive tests, however, provided significant incremental prognostic power (Fig. 6). The increment associated with exercise ECG averaged 14 f 10% (72 - 58, p = 0.05) and that associated with scintigraphy averaged 14 f 8% (86 72, p = 0.01). As a result, exercise ECG provided additional prognostic stratification in these patients (event rate = 3% vs 1470, p = 0.04) (Fig. 7). Moreover, scintigraphy provided further prognostic stratification in patients with positive exercise ECGs (event rate = 0% vs 24%. p = 0.06) and in patients with discordant exercise ECGs (event rate = 2% vs 1670, p = 0.008) (Fig. 8).
Discussion
cians always have knowledge of at lease some aspects of the history and physical examination before they have knowledge of a test response. Despite this rather obvious fact, however, technologic assessment is often performed using analytic models that ignore the amount of information already known before testing or its ease of acquisition. For example, a group of investigators recently used stepwise discriminant function analysis to assess the utility of exercise radionuclide ventriculography in predicting subsequent coronary events soon after acute myocardial infarction.z5 Of 15 clinical, ECG and scintigraphic variables, discriminant analysis identi-
Our study confirms an earlier report indicating that the prior probability of disease is the most important descriptor of prognosis in patients referred for stress testing because of suspected CAD,l’ and shows that exercise ECG and scintigraphy each have additional prognostic power only in certain clinical subsets. These findings have direct relevance to the development of optimal testing strategies. The concept of incremental information: Clinical information is never interpreted in isolation. Physi-
”
PERFUSION SCINTIGRAPHY
3c )-
0.5 POSITIVE
FALSE
1
1.0 RATE
FIGURE 6. Receiver operatlng characteristic curves in patients with abnormal rest electrocardiographic findings. From the lowest to the highest, these curves represent clinical history (H), clinical history plus exercise electrocardiogram (H+E) and clinical history plus electrocardiography plus thallium scintigraphy (H+E+T).
*P-x
20 EVENT RATE (%I
20 1c
-
EXERCISE ELECTROCARDIOGRAPHY PRE-EXERCISE DISEASE PROSASILITY “=
-
f
60 12
2
21 39
7
2 F
+
HGH 13
p-=.05
36
FIGURE 5. Perfuslon scintigraphy and prognosis. Patients are grouped by disease probability (Fig. 3), by exercise electrocardlographic response (Fig. 4) and by scintlgraphic response. Sclntlgraphic findings were considered normal If anything less than moderate hypoperfusion was observed and were considered abnormalif moderate or severe hypoperfusion was observed. At low probabillty, scintlgraphy provided no additional prognostic discrimination. At intermediate probability, discrimination was improved for patients with positive (-I) exercise electrocardiographic responses, and at high probability, discrimination was improved for patients with dlscordant (A) exercise electrocardiographic responses. Htgh-probability patients with negatlve (-) electrocardiographic responses were at low risk and high-probability patients with positive (+) responses were at high risk, regardless of the sclntlgraphlc response.
10
5 2
rl 0
n=
+ EXERCISE ELECTROCARDIOGRAPHY 70 ID3 35
FIGURE 7. Exercise electrocardiography with abnormal rest electrocardiographic grouped as In Figure 4. Event rates abnormal exercise electrocardiographic
and prognosis in patients findings. Patlents are increased with increasingly response.
February
fied only the latter as providing significant prognostic information. On the basis of this analysis, the investigators concluded that exercise radionuclide ventriculography “is a highly sensitive means for classifying patients. . .according to the likelihood of having cardiac events” in the early months after myocardial infarction.25 When we performed discriminant analysis on a similar database, we too found the exercise ejection fraction to be the only variable with statistically significant prognostic value 26; but when we analyzed the data according to a clinically realistic hierarchy whereby the individual variables were considered for inclusion in the very same order in which they actually become available to the physician (first, the historical variables; second, the exercise ECG variables; third, the rest/exercise radionuclide variables), the exercise radionuclide variables no longer provided a significant increment of prognostic information.26 Similarly, Currie et alz7 showed that neither exercise ECG nor radionuclide ventriculography alone provided significantly more diagnostic information then did clinical history; only when the 2 stress tests were considered together was accuracy improved. Previous studies document the utility of clinical history,2*Jg exercise ECG1-12 and perfusion scintigraphy13-15 in assessing the prognosis of patients with suspected CAD, but they fail to recognize the incremental hierarchical nature of such information. The fact that historical information and rest ECG data are available to the clinician before exercise ECG and scintigraphic data has important implications for optimal utilization of these procedures. A testing strategy based on incremental information: A strategy for prognostic testing can be developed based on the incremental prognostic power of the in-
1, 1987
THE AMERICAN
JOURNAL
OF CARDIOLOGY
Volume
59
275
formation and the cost of obtaining that information. We shall here limit our discussion to monetary costs, although other costs are also likely to be important (Fig. 9). Since we assume that a brief history suitable for estimation of disease probability and a rest ECG will be recorded in all patients, only the cost of performing additional noninvasive tests will be considered. Perfusion scintigraphy performed in conjunction with exercise ECG is approximately 3 times more expensive than exercise ECG alone (about $750 and $250, respectively). The total cost of testing our entire population with both noninvasive tests, therefore, was $750 times 1,659, or $1,244,250. The strategy proposed in Figure 10 is based on the monetary costs noted above and on the empiric observations reported herein. At each decision point, additional testing is recommended only if such testing is capable of producing “significant” stratification of the negative and positive responders into appropriately low- and high-risk groups. We used the conventional threshold of p = 0.05 to define “significant” stratification, and we chose an event rate of 5% as the threshold between low and high risk. For example, in the case of high-disease-probability patients with positive ECG findings, there was an apparent difference between the event rates of 25% and 9% for patients with normal and abnormal scintigraphic findings, respectively. However, the difference was not significant (p = 0.14, and while 25% represented an unacceptably high risk, 9% did not represent an acceptably low risk. According to our strategy, therefore, the entire group of patients is considered at high risk (event rate = 19%], and further noninvasive testing is not recommended. Patients with normal rest ECG findings are first classified by disease probability. Patients with low
SCINTIGRAPHY 20
EVENT RATE (%I
10
ECG PRE-EXERCISE
@
EXERCISE ELECTRocARDKx+IAPHYO
I “=
41
29
66
37
FIGURE 9. Perfusion scintlgraphy and prognosls In patients wlth abnormal rest electrocardlographlc findings. Patients are grouped as in Figure 5. Prognostic discrimination was improved for patlents wlth discordant (k) exercise electrocardiographic responses.
DISEASE
PROBABILITY
FIGURE 9. The monetary cost of a uniform approach to prognostic assessment (exercise electrocardiography [ECG] and scintlgraphy in all patients) is compared to that of a sequential testing strategy. Sclntigraphy (which includes exercise ECG) is assumed to cost $750 and exercise ECG alone is assumed to cost $250. A sequential testing strategy is cost-effective in patients with normal ECG flndings. There is no savings in patients wlth abnormal rest electrocardiographic findings.
276
INCREMENTAL
ACCURACY
OF CORONARY
ARTERY
DISEASE
PROGNOSIS
probability are not further tested because they are at low risk. Patients with intermediate probability are referred for exercise ECG and those with a negative response (no ST depression at a normal level of stress] or a discordant response (no ST depression at a reduced level of stress or ST depression at a normal level of stress) are not further tested because they also are at low risk. Those with a positive exercise ECG response (ST depression at a reduced level of stress] are referred for scintigraphy. Patients with normal scintigraphic responses are considered to be at low risk and patients with abnormal scans are considered to be at high risk. Since only 57 (l2%] intermediate-diseaseprobablility patients proceed to scintigraphic testing, cost is reduced by $204,000. Similarly, all patients with high disease probability and normal rest ECG findings are referred for exercise ECG, and those with negative and positive responses are not further tested because they are at low and high risk, respectively, regardless of the subsequent scintigraphic response. In contrast, patients with discordant exercise ECG responses are referred for scintigraphy and defined as being at low and high risk according to the scintigraphic response. Only 99 (47%) patients with
NORMAL RESTING ELECTROCARDIOGRAM (SequentId TestIngI
mscaRDuIT
HEaATM
PERFUSION SCNTKYIAPNY
POSITlYE -ANT I
N(UTM
FERNSION SCINTIGRAPHY
I #aNmNu
- 1'I v 1
NnAL
HIGH RISK I
v 1 LOW
RISK
ABNORMAL RES77NG ELECTROCARDIOGRAM OJnlformRstlngl 1 EXERCISE-
high probability would be so referred for scintigraphy, reducing the cost of testing by $3l,OOO. All patients with abnormal rest ECG responses are referred for exercise ECG and scintigraphy. This is not associated with a cost reduction. As a result of this strategy a high-risk group amounting to 11% of the total population is defined, while the total cost of testing is reduced by $795,500 (64%). Highrisk patients so defined are presumed to be appropriate candidates for more aggressive therapy (coronary angioplasty or bypass surgery], while low-risk patients might be treated more conservatively. Limitations: Certain limitations of our study design and analysis should be noted. First, the design was necessarily retrospective, and not all variables with potential prognostic importance were considered. Second, event rates were sometimes estimated from relatively small subsets, and the confidence associated with these estimates was limited accordingly. For example, only 1 event occurred in the 12 patients with high disease probability with normal exercise ECG responses and abnormal scintigraphic findings. Because of the small sample size, however, the 95% confidence interval associated with this event rate of 8% ranges from 0 to 39%. Third, the cutpoints used to define patient subsets with respect to disease probability and test response were arbitrary, and despite the availability of computer decision aids such as those used by us to calculate disease probability, physicians are likely to perform such estimation tasks subjectively and to thereby incorporate a host of additional factors not considered in the present analysis. Fourth, the statistical threshold of p = 0.05 used to define significant prognostic stratification, and the event rate threshold of 5% used to define high- and low-risk groups are both arbitrary. Changes in these thresholds could materially change the recommendations that derive from the strategic model. Clinical application: To use this model, then, each physician must select his own thresholds for acceptable and unacceptable risk and cost. He or she can then use these thresholds to decide on the need for further testing based on the available empiric data and on the rules embodied in the model. The model thus provides the physician with an explicit framework within which to exercise his clinical judgment. Selective use of exercise testing procedures according to our strategic model thereby stratifies individual patient risk for subsequent coronary events over a full order of magnitude at a 64% reduction in monetary cost.
1
References
I FIGURE
HIGH
RISK
10. A proposed
LOW
rational
testing
RISK
strategy.
I
1. Bruce RA. DeRouen T, Peterson DR. Irving JB. Chinn N, Blake B. Hofer V. Noninvasive predictors of sudden cardiac death in men with coronary artery disease. Am J Cardiol 1977;39:833-840. 2. Epstein SE. Value and limitations of electrocardiographic response to exercise in the assessment of patients with coronary artery disease. Controversies in cardiology. Am J Cardiol 1978:42:667-674. 3. McNeer JF. Margolis JR, Lee KL, Kisslo JA. Peter RH, Kong Y. Behar VS. Wallace AG. M&ants CB, Rosati RA. The role of the exercise test in the evaluation of patients for ischemic heart disease. Circulation 1978;57:64-70. 4. Hammermeister KE, DeRouen TA. Dodge HT. Variables predictive of survival in patients with coronary disease. Selection by univariate and multivariate analyses from the clinical, electrocardiographic, exercise, arteriographic, and quantitative qngiographic evaluation. Circulation 1979;59:421429.
February
5. Ivanova
LA, Mazur
NA. Smimova TM, Sumarokov AB. Nazarenko VA. exercise testing and ambulatory monitoring to identify patients with ischemic heart disease at high risk of sudden death. Am J Cardiol 1986;45:113.&1138. 6. Harris P]. Lee KL. Harrell FE, Behar VX, Rosati RA. Outcome in medically treated coronary artery disease. Ischemiti events non-fatal infarction and death. Circulation 1986;62:716-726. 7. Kennedy HL. Comparison of ambulatory electrocardiography and exercise testing. Am J Cardiol 1961;47:1359-1365. 8. Schneider RM. Seaworth JF. Dohrmann ML, Lester RM. Phillips HR. Bashore TM, Baker JT. Anatomic and prognostic implications of an early positive treadmill exercise test. Am J Cardiol 1962;50:682-688. 9. Gohlke H, Samek L. Betz P. Roskamm H. Exercise testing provides additional prognostic information in angiogmphically defined subgroups of patients with coronary artery disease. Circulation 1983;68:979-985. 10. Weiner DA, McCabe CH. Ryan TJ. Prognostic assessment of patients with coronary artery disease by exercise testing. Am Heart J 1983;105:749-755. 11. CASS Principal Investigators and their Associates. Coronary Artery Surgery Study [CASS). A randomized trial of coronary artery bypass surgery. Survival data. Circulation 1983;68:939. 12. Weiner DA Ryan TJ. McCabe CH. Chaitman BR. Sheffield LT. Ferguson JC. Fisher LD. Tristani F. Prognostic importance of a clinical profile and exercise test in medicallv treated wtients with coronary artery disease. IACC 1984;3:772-779. 13. Ladenheim ML, Pollock BH, Rozanski A. Berman DS. Staniloff HM, Forrester IS, Diamond GA. Extent and severity of myocardial hypoperfusion as predictors of prognosis in patients with suspected coronary artery disease. JACC 1986:7:464-471. 14. Brown KA. Boucher CA, Okada RD. Guiney TE, Newell JB. Strauss HW. Pohost GM. Prognostic value of exercise thallium-281 imaging in patients presenting for evaluation of chest pain. JACC 1983;1:994-1001. 15. Pamelia FX, Gibson RS. Watson DD. Craddock GB. Sirowatka J. Beller GA. Prognosis with chest pain and normal thallium-201 exercise scintigrams. Am J Cardiol 1985;55:920-928. 16. Diamond GA. A clinically relevant classification of chest discomfort. JACC 1983;1:574-575. 17. Diamond GA, Staniloff HM, Forrester JS. Pollock BH, Swan HJC. ComElectrocardiographic
I
I
1. 1987
THE AMERICAN
JOURNAL
OF CARDIOLOGY
Volume
59
277
puter-assisted diagnosis in the noninvasive evaluation of patients with suspected coronary artery disease. JACC 1983;1:444-445. 18. Staniloff HS. Forrester JS, Diamond GA, Pollock BH. Swan HJC. The “CABG patch”: classification of coronary artery bypass surgical crossovers in medical studies (abstr). CIin Res 1984;32:228A. 19. Dixon WJ. BMDP Statistical Software. Berkeley. CA: University of CaIifornia Press, 1981:330-341. 20. Hanley JA. McNeil BJ. The meaning and use of the area under o receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36. 21. McNeil Bj. Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (AOC] curves. Med Decis Making 1984;4:137-150. 22. Rozanski A, Diamond GA, Forrester JS, Berman DS. Morris D, Jones R, Okada R. Freeman M. Swan HJC. Should the intent of testing influence its interpretation? JACC 1986;7:17-24. 23. Diamond GA. ROC steady: a receiver operating characteristic curve which is invariant relative to selection bias. Med Decis Making, in press. 24. Hanley JA, NcNeiI BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983:148:839-843. 25. Corbett JR, Dehmer GJ. Lewis SE, Woodward W, Henderson E. Parkey RW. Blomquist CG, WilIerson JT. The prognostic value of submaximal exercise testing with radionuclide ventriculography before hospital discharge in patients with recent myocardial infarction. Circulation 1981:64;535-544. 26. Staniloff HM. Diamond GA, Forrester JS, Pollock BH. Berman DS, Swan HJC. The incremental information boondoggle: when a test seems powerful but isn’t. Circulation 1982;66:184. 27. Currie PH. Kelly MJ. Harper RW. Federman J, Kalff V. Anderson ST, Pitt BA. fncremental value of clinical assessment, supine exercise electrocardiography, and biplane exercise radionuclide ventriculography in the prediction of coronary artery disease in men with chest pain. Am J CardioI1983;52;927935. 28. Cole JP, Ellestad MH. Significance of chest pain during treadmill exercise: correlation with coronary events. Am J Cardiol 1876;41:227-232. 29. Cohn PF. Harris P. William BH, Rosati RA. Rosenbaum P. Watemaux C. Prognostic importance of angina1 symptoms of angina1 symptoms in angiogmphically defined coronary artery disease. Am J Cardiol 1981:487:223237.