- Email: [email protected]
Understanding the impact and challenges of secondary data analysis
Urologic Oncology: Seminars and Original Investigations ] (2017) ∎∎∎–∎∎∎
Seminars article
Understanding the effect and challenges of secondary data analysis Quoc-Dien Trinh, M.D.* Division of Urological Surgery, Center for Surgery and Public Health, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA Received 2 November 2017; accepted 6 November 2017
Abstract Secondary data analysis is commonly defined as the use of datasets, which were not collected for the purpose of the scientific hypothesis being tested. Examples of datasets range from private insurance claims to nationally administered health surveys. The use of secondary data confer several benefits, most notably by eliminating many of the financial and logistical obstacles related to primary data collection. The issues in using secondary data to answer important clinical and health policy questions are complex, but with appropriate and rigorous approaches there is an opportunity to produce high-effect research which can improve the care of patients with urologic malignancies. r 2017 Elsevier Inc. All rights reserved.
Keywords: Health services research; Health policy; Secondary data analysis; Methodology; Medicare
The use of secondary data in medical research has grown tremendously in recent years. Secondary data analysis is commonly defined as the use of datasets, which were not collected for the purpose of the scientific hypothesis being tested. Examples range from private insurance claims to nationally administered health surveys. The use of secondary data confer several benefits, most notably by eliminating many of the financial and logistical obstacles related to primary data collection. Although technical considerations previously limited the number of capable investigators to a small pool of individuals with specialized resources, several events have led to the “democratization” of health services research, and specifically in the realm of secondary data analysis: (1) significant efforts by large organizations to create and maintain easy-to-use, relatively inexpensive datasets, especially for research in oncology, (2) advances in statistical software—whereas in the past, complex data manipulation required significant coding skill and analytical time, some of these techniques have been reduced to a “click-of-a-button” in a graphical software suite, and (3) increased education and awareness in the field of health services research, led by pioneer institutions and mentors. What is more, many of the most urgent issues in health care author. Tel.: þ1-617-525-7350; fax: þ1-617-525-6348. E-mail address: [email protected] *Corresponding
https://doi.org/10.1016/j.urolonc.2017.11.003 1078-1439/r 2017 Elsevier Inc. All rights reserved.
today involve topics such as uncontrolled costs, disparities in outcomes, and uneven quality of care—issues, which have clear relevance for health policy-makers and which are often better studied using retrospective data rather than standard prospective trials [1]. On the other hand, investigators (and ultimately, reviewers and editors) need to recognize the caveats of secondary data analysis. It goes without saying that the scientific method must be followed. “Data mining” for statistically significant trends is rarely appropriate and investigators should be conducting research with clear, a priori hypotheses. Additionally, each type of secondary data come with its own limitations and biases; authors should take care not to inadvertently make the rhetorical jump from observational descriptions to inferred causality. This is especially true in studies addressing clinical management, which must be carefully interpreted as a “hypothesisgenerating,” not hypothesis proving. For this special issue of Urologic Oncology: Seminars and Original Investigations, we reached out to researchers with a track record of innovative and impactful publications using secondary data sources. It is our hope that these seminars will provide methodological, logistical and scientific insights for secondary data analysis in urologic oncology. We anticipate that these works will serve as a starting point for researchers and trainees alike, as they
2
Q.-D. Trinh / Urologic Oncology: Seminars and Original Investigations ] (2017) ∎∎∎–∎∎∎
navigate the challenges and limits of secondary data analysis in urologic oncology. The first seminar article by Cole et al. [2] provides an overview of common types of secondary data. The foundation of any successful research study utilizing secondary data analysis is the choice of an appropriate data source. This article emphasizes the critical importance of understanding the capabilities and the strengths and weaknesses of each type of data and need to select the right dataset to answer a clinical or policy question. When such datasets do not exist, investigators have resorted to creative linkages of datasets with a common identifier. Finally, another key message here is that a broad understanding of the limitations of a particular dataset is essential to provide the appropriate interpretation of study findings. Once the data source are selected, a statistical and analytic plan must be undertaken to address the unique sources of bias and to ensure the validity of results. In the second seminar article by Maxine Sun and Stuart Lipsitz, the authors, therefore, provide prospective investigators with a starting guide about the methodological intricacies in secondary data analysis [3]. The analytic plan should be tailored to both the question at hand and the data source being used. In the third seminar article by Serrell et al. [4], the authors synthetize the data on comparative effectiveness of prostate cancer treatments that have relied on secondary data. Despite the publication of results from the high-profile ProtecT trial last year, level 1 evidence on treatments for localized prostate cancer are scarce and limited by many technical considerations, none the least the constant evolution of prostate cancer diagnosis and treatment, as well as the indolent nature of this disease even in its most aggressive state, which adjudicates any study from 15 years ago to be limited in its scope. In the fourth seminar article by Edwards et al. [5], the authors review research using secondary data on nonprostate genitourinary malignancies. Again, a common theme here is the use of secondary data to inform the (many) questions where higher quality level 1 evidence does not exist. In many cases, such hypothesis-generating articles have led to proposal for prospective trials, highlighting the potential impact of secondary data analysis to change clinical practice. Finally, in the fifth seminar article by Wang et al. [6], the authors review the methodology and the landmark studies
assessing cost and cost-effectiveness using secondary data in urologic oncology. Given the current focus on cost containment, such studies comparing the value of competing strategies are gaining interest and impact in urologic research, and beyond that, have the ability to assist stakeholders in making policy changes, such as reversing the relative-value unit attribution for robot-assisted procedures. The issues in using secondary data to answer important clinical and health policy questions are complex, but with appropriate and rigorous approaches there is an opportunity to produce high-effect research which can improve the care of patients with urologic malignancies. The perspectives in these seminars will provide a starting point for those seeking to undertake studies using secondary data analysis and for those hoping to gain a better understanding of the literature published using these techniques. Acknowledgments Quoc-Dien Trinh is supported by an unrestricted educational grant from the Vattikuti Urology Institute, a Clay Hamlin Young Investigator Award from the Prostate Cancer Foundation and a Genentech Bio-Oncology Career Development Award from the Conquer Cancer Foundation of the American Society of Clinical Oncology. References [1] Trinh QD, Cole AP, Dasgupta P. Weighing the evidence from surgical trials. BJU Int 2017;119:659–60. [2] Cole AP, Friedlander DF, Trinh QD. Secondary data sources for health services research in urologic oncology. Urol Oncol 2017, [this issue]. [3] Sun M, Lipsitz S. Comparative effectiveness research methodology using secondary data: a starting user’s guide. Urol Oncol 2017, [this issue]. [4] Serrell EC, Pitts D, Hayn MH, Beaule L, Hansen M, Sammon J. Review of the comparative effectiveness of radical prostatectomy, radiation therapy, or expectant management of localized prostate cancer in registry data. Urol Oncol 2017, [this issue]. [5] Edwards DC, Cahn DB, Smaldone MC, Kutikov A. Use of administrative data for comparative effectiveness research in the treatment of non-prostate genitourinary malignancies. Urol Oncol 2017, [this issue]. [6] Wang Y, Mossanen M, Chang SL. Cost and cost-effectiveness studies in urologic oncology using large administrative databases. Urol Oncol 2017, [this issue].
Seminars article
Understanding the effect and challenges of secondary data analysis Quoc-Dien Trinh, M.D.* Division of Urological Surgery, Center for Surgery and Public Health, Brigham and Women’s Hospital, Harvard Medical School, Boston, MA Received 2 November 2017; accepted 6 November 2017
Abstract Secondary data analysis is commonly defined as the use of datasets, which were not collected for the purpose of the scientific hypothesis being tested. Examples of datasets range from private insurance claims to nationally administered health surveys. The use of secondary data confer several benefits, most notably by eliminating many of the financial and logistical obstacles related to primary data collection. The issues in using secondary data to answer important clinical and health policy questions are complex, but with appropriate and rigorous approaches there is an opportunity to produce high-effect research which can improve the care of patients with urologic malignancies. r 2017 Elsevier Inc. All rights reserved.
Keywords: Health services research; Health policy; Secondary data analysis; Methodology; Medicare
The use of secondary data in medical research has grown tremendously in recent years. Secondary data analysis is commonly defined as the use of datasets, which were not collected for the purpose of the scientific hypothesis being tested. Examples range from private insurance claims to nationally administered health surveys. The use of secondary data confer several benefits, most notably by eliminating many of the financial and logistical obstacles related to primary data collection. Although technical considerations previously limited the number of capable investigators to a small pool of individuals with specialized resources, several events have led to the “democratization” of health services research, and specifically in the realm of secondary data analysis: (1) significant efforts by large organizations to create and maintain easy-to-use, relatively inexpensive datasets, especially for research in oncology, (2) advances in statistical software—whereas in the past, complex data manipulation required significant coding skill and analytical time, some of these techniques have been reduced to a “click-of-a-button” in a graphical software suite, and (3) increased education and awareness in the field of health services research, led by pioneer institutions and mentors. What is more, many of the most urgent issues in health care author. Tel.: þ1-617-525-7350; fax: þ1-617-525-6348. E-mail address: [email protected] *Corresponding
https://doi.org/10.1016/j.urolonc.2017.11.003 1078-1439/r 2017 Elsevier Inc. All rights reserved.
today involve topics such as uncontrolled costs, disparities in outcomes, and uneven quality of care—issues, which have clear relevance for health policy-makers and which are often better studied using retrospective data rather than standard prospective trials [1]. On the other hand, investigators (and ultimately, reviewers and editors) need to recognize the caveats of secondary data analysis. It goes without saying that the scientific method must be followed. “Data mining” for statistically significant trends is rarely appropriate and investigators should be conducting research with clear, a priori hypotheses. Additionally, each type of secondary data come with its own limitations and biases; authors should take care not to inadvertently make the rhetorical jump from observational descriptions to inferred causality. This is especially true in studies addressing clinical management, which must be carefully interpreted as a “hypothesisgenerating,” not hypothesis proving. For this special issue of Urologic Oncology: Seminars and Original Investigations, we reached out to researchers with a track record of innovative and impactful publications using secondary data sources. It is our hope that these seminars will provide methodological, logistical and scientific insights for secondary data analysis in urologic oncology. We anticipate that these works will serve as a starting point for researchers and trainees alike, as they
2
Q.-D. Trinh / Urologic Oncology: Seminars and Original Investigations ] (2017) ∎∎∎–∎∎∎
navigate the challenges and limits of secondary data analysis in urologic oncology. The first seminar article by Cole et al. [2] provides an overview of common types of secondary data. The foundation of any successful research study utilizing secondary data analysis is the choice of an appropriate data source. This article emphasizes the critical importance of understanding the capabilities and the strengths and weaknesses of each type of data and need to select the right dataset to answer a clinical or policy question. When such datasets do not exist, investigators have resorted to creative linkages of datasets with a common identifier. Finally, another key message here is that a broad understanding of the limitations of a particular dataset is essential to provide the appropriate interpretation of study findings. Once the data source are selected, a statistical and analytic plan must be undertaken to address the unique sources of bias and to ensure the validity of results. In the second seminar article by Maxine Sun and Stuart Lipsitz, the authors, therefore, provide prospective investigators with a starting guide about the methodological intricacies in secondary data analysis [3]. The analytic plan should be tailored to both the question at hand and the data source being used. In the third seminar article by Serrell et al. [4], the authors synthetize the data on comparative effectiveness of prostate cancer treatments that have relied on secondary data. Despite the publication of results from the high-profile ProtecT trial last year, level 1 evidence on treatments for localized prostate cancer are scarce and limited by many technical considerations, none the least the constant evolution of prostate cancer diagnosis and treatment, as well as the indolent nature of this disease even in its most aggressive state, which adjudicates any study from 15 years ago to be limited in its scope. In the fourth seminar article by Edwards et al. [5], the authors review research using secondary data on nonprostate genitourinary malignancies. Again, a common theme here is the use of secondary data to inform the (many) questions where higher quality level 1 evidence does not exist. In many cases, such hypothesis-generating articles have led to proposal for prospective trials, highlighting the potential impact of secondary data analysis to change clinical practice. Finally, in the fifth seminar article by Wang et al. [6], the authors review the methodology and the landmark studies
assessing cost and cost-effectiveness using secondary data in urologic oncology. Given the current focus on cost containment, such studies comparing the value of competing strategies are gaining interest and impact in urologic research, and beyond that, have the ability to assist stakeholders in making policy changes, such as reversing the relative-value unit attribution for robot-assisted procedures. The issues in using secondary data to answer important clinical and health policy questions are complex, but with appropriate and rigorous approaches there is an opportunity to produce high-effect research which can improve the care of patients with urologic malignancies. The perspectives in these seminars will provide a starting point for those seeking to undertake studies using secondary data analysis and for those hoping to gain a better understanding of the literature published using these techniques. Acknowledgments Quoc-Dien Trinh is supported by an unrestricted educational grant from the Vattikuti Urology Institute, a Clay Hamlin Young Investigator Award from the Prostate Cancer Foundation and a Genentech Bio-Oncology Career Development Award from the Conquer Cancer Foundation of the American Society of Clinical Oncology. References [1] Trinh QD, Cole AP, Dasgupta P. Weighing the evidence from surgical trials. BJU Int 2017;119:659–60. [2] Cole AP, Friedlander DF, Trinh QD. Secondary data sources for health services research in urologic oncology. Urol Oncol 2017, [this issue]. [3] Sun M, Lipsitz S. Comparative effectiveness research methodology using secondary data: a starting user’s guide. Urol Oncol 2017, [this issue]. [4] Serrell EC, Pitts D, Hayn MH, Beaule L, Hansen M, Sammon J. Review of the comparative effectiveness of radical prostatectomy, radiation therapy, or expectant management of localized prostate cancer in registry data. Urol Oncol 2017, [this issue]. [5] Edwards DC, Cahn DB, Smaldone MC, Kutikov A. Use of administrative data for comparative effectiveness research in the treatment of non-prostate genitourinary malignancies. Urol Oncol 2017, [this issue]. [6] Wang Y, Mossanen M, Chang SL. Cost and cost-effectiveness studies in urologic oncology using large administrative databases. Urol Oncol 2017, [this issue].