- Email: [email protected]
Merits of Data Sharing
JOURNAL OF THE AMERICAN COLLEGE OF CARDIOLOGY
VOL. 70, NO. 14, 2017
ª 2017 BY THE AMERICAN COLLEGE OF CARDIOLOGY FOUNDATION
ISSN 0735-1097/$36.00
PUBLISHED BY ELSEVIER
Letters Merits of Data Sharing
treatment. We then determined which variables were
The Digitalis Investigation Group Trial
which were reported in subsequent publications.
reported in the trial’s original publication (2), and Identifications
were
made
by
one
author
and
confirmed by a second author. Data sharing is important to maximize what can be
After the trial’s main publication in 1997, there
learned from clinical trials (1). The DIG (Digitalis
were no subsequent publications through 2002. Since
Investigation Group) trial is ideal to assess the effects
2002 until December 31, 2016, 75 studies have been
of data sharing. A federally funded trial conducted
published, of which 41 (55%) were conducted by
from 1991 to 1995 and published in 1997, the DIG trial
outside investigators. Outside investigators pub-
randomized 6,800 people with stable heart failure to
lished 17 studies from 2002 to 2009, and 24 from 2010
digoxin or placebo (2). Individual patient-level data
to 2016. Trial investigators published 28 studies from
were made available from 2000 to 2002 through the
2002 to 2009 and 6 from 2010 to 2016, but none since
National Heart, Lung, and Blood Institute’s Limited
2014 (Figure 1).
Access Datasets program. In 2009, data were made
Of the 41 studies by outside investigators, 5 were
widely available through the National Institutes of
published in journals with an impact factor $10. The
Health’s Biologic Specimen and Data Repository
median citations per article-year were 4.8 (IQR: 2.2
Information Coordinating Center (BioLINCC). We
to 7.7). The top 2 cited studies had 652 and 637
sought to determine the contributions of outside in-
citations. Of 34 studies by trial investigators, 7 were
vestigators to the trial’s publication yield as a metric
published in journals with an impact factor $10. The
for the benefits of data sharing, and compare publi-
median citations per article-year were 6.9 (IQR: 3.8
cations by trial investigators with those of outside
to 11.2). The top 2 cited studies had 424 and 338
investigators. We also assessed the use of various
citations.
variables collected in the trial to measure the extent to which the data have been harvested.
We identified 230 distinct variables collected in the DIG trial, of which 58 (25%) were reported in the
We searched PubMed and used BioLINCC’s data
initial publication. As of December 31, 2016, 149 (65%)
repository to identify all published studies that used
variables have been reported in various studies
the DIG trial data. We compared the number of
published using the trial data.
studies involving $1 trial investigator with those
In this study, we found that outside investigators
involving only outside investigator from 2002 to 2009
have authored a significant proportion of publications
(pre-BioLINCC availability) and 2010 to 2016 (post-
from the DIG trial and have been instrumental in
BioLINCC availability). To represent the timeline
harvesting the data. Based on the timeline of publi-
and impact of the studies, each article’s publication
cations, making trial data publicly available seems to
date
recently
spur publications by both outside and trial investi-
available impact factor of the publishing journal. We
gator groups. Studies by outside investigators have
calculated citations per article-year by dividing the
produced
number of citations of each study by the number of
example, digoxin was found to increase mortality
years since its publication. Median citations per
in women but not men (3), and was associated
article-year (interquartile range [IQR]) were calcu-
with increasing mortality with higher serum concen-
lated for studies by trial investigators and outside
tration (4).
was
plotted
against
the
most
investigators to assess the impact of publications by the 2 groups.
key
practice-changing
insights;
for
The impact factors of the journals and the citations among trial investigators and outside in-
To assess the data utilization from the DIG trial, we
vestigators were similar, speaking for both the
identified the variables collected from the data
quality and the rigor of both groups of investigators.
dictionary and included those related to patient
However, one-third of the variables collected in the
demographics, assessment, follow-up, events, and
trial remain unreported, indicating that there may
JACC VOL. 70, NO. 14, 2017
Letters
OCTOBER 3, 2017:1825–32
F I G U R E 1 Timeline of the Studies Published Using the DIG Trial Data According to the Impact Factor of the Publishing Journals
70 Original Trial Publication 60
50
Impact Factor
1826
40
30
20
10
0 1/1/1997
1/1/1999
1/1/2001
1/1/2003 1/1/2005 1/1/2007 1/1/2009
1/1/2011
1/1/2013
1/1/2015
Publication Dates Time Period when Data were Made Available
Studies by Trial Investigators
Studies by Independent Researchers
DIG ¼ Digitalis Investigation Group. April 2000 to February 2002, data available through Limited Access Datasets program. October 2009, data released on National Institutes of Health’s Biologic Specimen and Data Repository Information Coordinating Center.
still be information remaining to be harvested from the DIG trial. Although our study relied on metrics of publication and citation that may not perfectly measure the importance of publications, it reveals the potential for data sharing to facilitate knowledge generation by enabling investigators who were not part of a trial to make substantive contributions to the ultimate yield of the effort. Suveen Angraal, MBBS Joseph S. Ross, MD, MHS Sanket S. Dhruva, MD Nihar R. Desai, MD, MPH John W. Welsh, BA *Harlan M. Krumholz, MD, SM *Yale Center for Outcomes Research and Evaluation 1 Church Street Suite 200 New Haven, Connecticut 06510 E-mail: [email protected] http://dx.doi.org/10.1016/j.jacc.2017.07.786
Please note: Dr. Desai is funded by grant K12 HS023000-03 from the Agency for Healthcare Research and Quality. Dr. Ross is supported by the Laura and John Arnold Foundation for the Collaboration on Research Integrity and Transparency at Yale and by the Food and Drug Administration for the Center for Excellence in Regulatory Science and Innovation program. The sponsors did not play a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Dr. Ross has received research support from the Blue Cross Blue Shield Association. Drs. Ross, Desai, and Krumholz have received research grant support from Medtronic and Johnson & Johnson (Janssen), through Yale, to develop methods of clinical trial data sharing. Drs. Ross and Krumholz have received research grant support from Medtronic and the Food and Drug Administration, through Yale, to develop methods for post-market surveillance of medical devices. Dr. Krumholz has contracts with the Centers for Medicare & Medicaid Services; has served as the chair of a cardiac scientific advisory board for UnitedHealth; has served as a participant/participant representative of the IBM Watson Health Life Sciences Board; has served on the advisory board for Element Science; has served on the physician advisory board for Aetna; has served on the open trials advisory board for the Laura and John Arnold Foundation through the Center for Open Science; and is the founder of Hugo, a personal health information platform. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
REFERENCES 1. Ross JS, Krumholz HM. Ushering in a new era of open science through data sharing: the wall must come down. JAMA 2013;309:1355–6. 2. The Digitalis Investigation Group. The effect of digoxin on mortality and morbidity in patients with heart failure. N Engl J Med 1997;336: 525–33.
JACC VOL. 70, NO. 14, 2017
Letters
OCTOBER 3, 2017:1825–32
3. Rathore SS, Wang Y, Krumholz HM. Sex-based differences in the effect of digoxin for the treatment of heart failure. N Engl J Med 2002;347: 1403–11. 4. Rathore SS, Curtis JP, Wang Y, Bristow MR, Krumholz HM. Association of serum digoxin concentration and outcomes in patients with heart failure. JAMA 2003;289:871–8.
pressures in patients with HF have been shown to be associated with increased lung water density measured with magnetic resonance imaging (2); however, the relationship between lung water content and peak V O 2 has not been evaluated. We measured peak V O2, lung water density, and cardiac structure and function in clinically stable outpatients,
Subclinical Pulmonary Edema Is Associated With Reduced Exercise Capacity in HFpEF and HFrEF
older patients with HF with preserved ejection fraction (HFpEF) (n ¼ 23) or HF with reduced ejection fraction (HFrEF) (n ¼ 23), and 16 agematched healthy control subjects from the Alberta HEART (Heart Failure Etiology and Analysis Research Team) study (3). Cardiopulmonary exercise testing was performed in the upright position on a cycle ergometer during
A cardinal feature of heart failure (HF) is reduced
which time V O 2, heart rate, and blood pressure were
exercise tolerance (peak oxygen uptake [VO2]) that is
measured. All HF patients were classified into those
associated with dyspnea and increased left ventricu-
above (HF Group A) or below (HF Group B) a peak V O2
lar (LV) filling pressure (1). Elevated resting LV filling
of 15.4 ml/kg/min, which has been previously
T A B L E 1 Participant Characteristics, Rest and Exercise Hemodynamics, and Lung Water Content
Control Group (n ¼ 16)
Age, yrs Male
HF Group A (n ¼ 23)
HF Group B (n ¼ 23)
ANOVA p Value
HFpEF (n ¼ 23)
HFrEF (n ¼ 23)
67 9
65 9
72 11
0.07
71 10
66 10
10 (62.5)
17 (73.9)
14 (60.9)
0.62
13 (56.5)
18 (78.3)
Height, cm
171 8
174 7
169 9
0.19
170 9
173 7
Weight, kg
79 13
91 14
92 19
0.01*†
94 19
89 13
1.92 0.20
2.11 0.20
2.00 0.21
0.02*†
2.06 0.23
2.05 0.20
26.9 3.7
30.7 2.8
30.1 4.2
31.3 3.3
29.7 4.4
14/9
9/14
23/0
0/23
BSA, m2 BMI, kg/m2 HFpEF/HFrEF NYHA functional class
0.003*† 0.41
I
12
8
9
11
II
8
10
9
9
III
3
5
5
3
62.5 7.0
46.7 10.1
49.0 16.9
56.9 9.1
38.3 11.0‡
LVEF, %
0.001*†
LVEDV index, ml/m2
75 19
89 31
84 30
0.31
71 18
103 32‡
LVESV index, ml/m2
29 10
49 21
46 32
0.04†
31 13
65 26‡
LV mass index, g/m2
59 11
69 13
72 20
0.06
65 13
75 18‡
Rest SBP, mm Hg
120 15
120 16
117 14
0.57
119 13
118 18
Rest DBP, mm Hg
69 9
74 11
68 10
0.64
76 9
73 9
Rest HR, beats/min
72 17
76 14
71 10
0.41
76 14
71 11
Peak SBP, mm Hg
176 19
146 27
140 24
<0.001*†
149 22
138 28
Peak DBP, mm Hg
84 6
79 12
76 10
0.04†
80 11
76 11
Peak HR, beats/min
146 24
137 26
112 23
<0.001†§
121 27
124 30
131 57
111 37
79 22
<0.001*†§
93 40
97 28
Peak VO2, ml/min/kg
24.3 9.0
19.1 3.1
13.1 1.8
<0.001*†§
15.8 6.3
17.6 4.9
Peak VO2, ml/min
1,912 734
1,792 392
1,218 314
<0.001*†§
1,462 484
1,477 388
Lung water density, %
16.6 2.0
16.7 2.8
20.6 3.9
<0.001†§
19.2 4.5
18.1 3.2
Peak workload, W
Values are mean SD, n (%), n/n, or n. Heart failure (HF) Group A: peak oxygen uptake (VO2) >15.4 ml/kg/min. HF Group B: peak VO2 <15.4 ml/kg/min. *Post hoc comparison for analysis of variance (ANOVA) <0.05 with Bonferroni correction: p < 0.05 between control group and HF Group A. †Post hoc comparison for ANOVA <0.05 with Bonferroni correction: p < 0.05 between control group and HF Group B. ‡p < 0.05 between HF with preserved ejection fraction (HFpEF) and HF with reduced ejection fraction (HFrEF). §Post hoc comparison for ANOVA <0.05 with Bonferroni correction: p < 0.05 between HF Group A and HF Group B. BMI ¼ body mass index; BSA ¼ body surface area; DBP ¼ diastolic blood pressure; HR ¼ heart rate; LV ¼ left ventricular; LVEDV ¼ left ventricular end-diastolic volume; LVEF ¼ left ventricular ejection fraction; LVESV ¼ left ventricular end-systolic volume; NYHA ¼ New York Heart Association; SBP ¼ systolic blood pressure.
1827
VOL. 70, NO. 14, 2017
ª 2017 BY THE AMERICAN COLLEGE OF CARDIOLOGY FOUNDATION
ISSN 0735-1097/$36.00
PUBLISHED BY ELSEVIER
Letters Merits of Data Sharing
treatment. We then determined which variables were
The Digitalis Investigation Group Trial
which were reported in subsequent publications.
reported in the trial’s original publication (2), and Identifications
were
made
by
one
author
and
confirmed by a second author. Data sharing is important to maximize what can be
After the trial’s main publication in 1997, there
learned from clinical trials (1). The DIG (Digitalis
were no subsequent publications through 2002. Since
Investigation Group) trial is ideal to assess the effects
2002 until December 31, 2016, 75 studies have been
of data sharing. A federally funded trial conducted
published, of which 41 (55%) were conducted by
from 1991 to 1995 and published in 1997, the DIG trial
outside investigators. Outside investigators pub-
randomized 6,800 people with stable heart failure to
lished 17 studies from 2002 to 2009, and 24 from 2010
digoxin or placebo (2). Individual patient-level data
to 2016. Trial investigators published 28 studies from
were made available from 2000 to 2002 through the
2002 to 2009 and 6 from 2010 to 2016, but none since
National Heart, Lung, and Blood Institute’s Limited
2014 (Figure 1).
Access Datasets program. In 2009, data were made
Of the 41 studies by outside investigators, 5 were
widely available through the National Institutes of
published in journals with an impact factor $10. The
Health’s Biologic Specimen and Data Repository
median citations per article-year were 4.8 (IQR: 2.2
Information Coordinating Center (BioLINCC). We
to 7.7). The top 2 cited studies had 652 and 637
sought to determine the contributions of outside in-
citations. Of 34 studies by trial investigators, 7 were
vestigators to the trial’s publication yield as a metric
published in journals with an impact factor $10. The
for the benefits of data sharing, and compare publi-
median citations per article-year were 6.9 (IQR: 3.8
cations by trial investigators with those of outside
to 11.2). The top 2 cited studies had 424 and 338
investigators. We also assessed the use of various
citations.
variables collected in the trial to measure the extent to which the data have been harvested.
We identified 230 distinct variables collected in the DIG trial, of which 58 (25%) were reported in the
We searched PubMed and used BioLINCC’s data
initial publication. As of December 31, 2016, 149 (65%)
repository to identify all published studies that used
variables have been reported in various studies
the DIG trial data. We compared the number of
published using the trial data.
studies involving $1 trial investigator with those
In this study, we found that outside investigators
involving only outside investigator from 2002 to 2009
have authored a significant proportion of publications
(pre-BioLINCC availability) and 2010 to 2016 (post-
from the DIG trial and have been instrumental in
BioLINCC availability). To represent the timeline
harvesting the data. Based on the timeline of publi-
and impact of the studies, each article’s publication
cations, making trial data publicly available seems to
date
recently
spur publications by both outside and trial investi-
available impact factor of the publishing journal. We
gator groups. Studies by outside investigators have
calculated citations per article-year by dividing the
produced
number of citations of each study by the number of
example, digoxin was found to increase mortality
years since its publication. Median citations per
in women but not men (3), and was associated
article-year (interquartile range [IQR]) were calcu-
with increasing mortality with higher serum concen-
lated for studies by trial investigators and outside
tration (4).
was
plotted
against
the
most
investigators to assess the impact of publications by the 2 groups.
key
practice-changing
insights;
for
The impact factors of the journals and the citations among trial investigators and outside in-
To assess the data utilization from the DIG trial, we
vestigators were similar, speaking for both the
identified the variables collected from the data
quality and the rigor of both groups of investigators.
dictionary and included those related to patient
However, one-third of the variables collected in the
demographics, assessment, follow-up, events, and
trial remain unreported, indicating that there may
JACC VOL. 70, NO. 14, 2017
Letters
OCTOBER 3, 2017:1825–32
F I G U R E 1 Timeline of the Studies Published Using the DIG Trial Data According to the Impact Factor of the Publishing Journals
70 Original Trial Publication 60
50
Impact Factor
1826
40
30
20
10
0 1/1/1997
1/1/1999
1/1/2001
1/1/2003 1/1/2005 1/1/2007 1/1/2009
1/1/2011
1/1/2013
1/1/2015
Publication Dates Time Period when Data were Made Available
Studies by Trial Investigators
Studies by Independent Researchers
DIG ¼ Digitalis Investigation Group. April 2000 to February 2002, data available through Limited Access Datasets program. October 2009, data released on National Institutes of Health’s Biologic Specimen and Data Repository Information Coordinating Center.
still be information remaining to be harvested from the DIG trial. Although our study relied on metrics of publication and citation that may not perfectly measure the importance of publications, it reveals the potential for data sharing to facilitate knowledge generation by enabling investigators who were not part of a trial to make substantive contributions to the ultimate yield of the effort. Suveen Angraal, MBBS Joseph S. Ross, MD, MHS Sanket S. Dhruva, MD Nihar R. Desai, MD, MPH John W. Welsh, BA *Harlan M. Krumholz, MD, SM *Yale Center for Outcomes Research and Evaluation 1 Church Street Suite 200 New Haven, Connecticut 06510 E-mail: [email protected] http://dx.doi.org/10.1016/j.jacc.2017.07.786
Please note: Dr. Desai is funded by grant K12 HS023000-03 from the Agency for Healthcare Research and Quality. Dr. Ross is supported by the Laura and John Arnold Foundation for the Collaboration on Research Integrity and Transparency at Yale and by the Food and Drug Administration for the Center for Excellence in Regulatory Science and Innovation program. The sponsors did not play a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication. Dr. Ross has received research support from the Blue Cross Blue Shield Association. Drs. Ross, Desai, and Krumholz have received research grant support from Medtronic and Johnson & Johnson (Janssen), through Yale, to develop methods of clinical trial data sharing. Drs. Ross and Krumholz have received research grant support from Medtronic and the Food and Drug Administration, through Yale, to develop methods for post-market surveillance of medical devices. Dr. Krumholz has contracts with the Centers for Medicare & Medicaid Services; has served as the chair of a cardiac scientific advisory board for UnitedHealth; has served as a participant/participant representative of the IBM Watson Health Life Sciences Board; has served on the advisory board for Element Science; has served on the physician advisory board for Aetna; has served on the open trials advisory board for the Laura and John Arnold Foundation through the Center for Open Science; and is the founder of Hugo, a personal health information platform. All other authors have reported that they have no relationships relevant to the contents of this paper to disclose.
REFERENCES 1. Ross JS, Krumholz HM. Ushering in a new era of open science through data sharing: the wall must come down. JAMA 2013;309:1355–6. 2. The Digitalis Investigation Group. The effect of digoxin on mortality and morbidity in patients with heart failure. N Engl J Med 1997;336: 525–33.
JACC VOL. 70, NO. 14, 2017
Letters
OCTOBER 3, 2017:1825–32
3. Rathore SS, Wang Y, Krumholz HM. Sex-based differences in the effect of digoxin for the treatment of heart failure. N Engl J Med 2002;347: 1403–11. 4. Rathore SS, Curtis JP, Wang Y, Bristow MR, Krumholz HM. Association of serum digoxin concentration and outcomes in patients with heart failure. JAMA 2003;289:871–8.
pressures in patients with HF have been shown to be associated with increased lung water density measured with magnetic resonance imaging (2); however, the relationship between lung water content and peak V O 2 has not been evaluated. We measured peak V O2, lung water density, and cardiac structure and function in clinically stable outpatients,
Subclinical Pulmonary Edema Is Associated With Reduced Exercise Capacity in HFpEF and HFrEF
older patients with HF with preserved ejection fraction (HFpEF) (n ¼ 23) or HF with reduced ejection fraction (HFrEF) (n ¼ 23), and 16 agematched healthy control subjects from the Alberta HEART (Heart Failure Etiology and Analysis Research Team) study (3). Cardiopulmonary exercise testing was performed in the upright position on a cycle ergometer during
A cardinal feature of heart failure (HF) is reduced
which time V O 2, heart rate, and blood pressure were
exercise tolerance (peak oxygen uptake [VO2]) that is
measured. All HF patients were classified into those
associated with dyspnea and increased left ventricu-
above (HF Group A) or below (HF Group B) a peak V O2
lar (LV) filling pressure (1). Elevated resting LV filling
of 15.4 ml/kg/min, which has been previously
T A B L E 1 Participant Characteristics, Rest and Exercise Hemodynamics, and Lung Water Content
Control Group (n ¼ 16)
Age, yrs Male
HF Group A (n ¼ 23)
HF Group B (n ¼ 23)
ANOVA p Value
HFpEF (n ¼ 23)
HFrEF (n ¼ 23)
67 9
65 9
72 11
0.07
71 10
66 10
10 (62.5)
17 (73.9)
14 (60.9)
0.62
13 (56.5)
18 (78.3)
Height, cm
171 8
174 7
169 9
0.19
170 9
173 7
Weight, kg
79 13
91 14
92 19
0.01*†
94 19
89 13
1.92 0.20
2.11 0.20
2.00 0.21
0.02*†
2.06 0.23
2.05 0.20
26.9 3.7
30.7 2.8
30.1 4.2
31.3 3.3
29.7 4.4
14/9
9/14
23/0
0/23
BSA, m2 BMI, kg/m2 HFpEF/HFrEF NYHA functional class
0.003*† 0.41
I
12
8
9
11
II
8
10
9
9
III
3
5
5
3
62.5 7.0
46.7 10.1
49.0 16.9
56.9 9.1
38.3 11.0‡
LVEF, %
0.001*†
LVEDV index, ml/m2
75 19
89 31
84 30
0.31
71 18
103 32‡
LVESV index, ml/m2
29 10
49 21
46 32
0.04†
31 13
65 26‡
LV mass index, g/m2
59 11
69 13
72 20
0.06
65 13
75 18‡
Rest SBP, mm Hg
120 15
120 16
117 14
0.57
119 13
118 18
Rest DBP, mm Hg
69 9
74 11
68 10
0.64
76 9
73 9
Rest HR, beats/min
72 17
76 14
71 10
0.41
76 14
71 11
Peak SBP, mm Hg
176 19
146 27
140 24
<0.001*†
149 22
138 28
Peak DBP, mm Hg
84 6
79 12
76 10
0.04†
80 11
76 11
Peak HR, beats/min
146 24
137 26
112 23
<0.001†§
121 27
124 30
131 57
111 37
79 22
<0.001*†§
93 40
97 28
Peak VO2, ml/min/kg
24.3 9.0
19.1 3.1
13.1 1.8
<0.001*†§
15.8 6.3
17.6 4.9
Peak VO2, ml/min
1,912 734
1,792 392
1,218 314
<0.001*†§
1,462 484
1,477 388
Lung water density, %
16.6 2.0
16.7 2.8
20.6 3.9
<0.001†§
19.2 4.5
18.1 3.2
Peak workload, W
Values are mean SD, n (%), n/n, or n. Heart failure (HF) Group A: peak oxygen uptake (VO2) >15.4 ml/kg/min. HF Group B: peak VO2 <15.4 ml/kg/min. *Post hoc comparison for analysis of variance (ANOVA) <0.05 with Bonferroni correction: p < 0.05 between control group and HF Group A. †Post hoc comparison for ANOVA <0.05 with Bonferroni correction: p < 0.05 between control group and HF Group B. ‡p < 0.05 between HF with preserved ejection fraction (HFpEF) and HF with reduced ejection fraction (HFrEF). §Post hoc comparison for ANOVA <0.05 with Bonferroni correction: p < 0.05 between HF Group A and HF Group B. BMI ¼ body mass index; BSA ¼ body surface area; DBP ¼ diastolic blood pressure; HR ¼ heart rate; LV ¼ left ventricular; LVEDV ¼ left ventricular end-diastolic volume; LVEF ¼ left ventricular ejection fraction; LVESV ¼ left ventricular end-systolic volume; NYHA ¼ New York Heart Association; SBP ¼ systolic blood pressure.
1827