- Email: [email protected]
How to do clinical research in cranio-maxillo-facial surgery
Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
Contents lists available at ScienceDirect
Journal of Cranio-Maxillo-Facial Surgery journal homepage: www.jcmfs.com
How to do clinical research in cranio-maxillo-facial surgery Maurice Y. Mommaerts a, Murray E. Foster b, Karsten K.H. Gundlach c, * a
Division of Maxillo-Facial Surgery, AZ St. Jan, Bruges, Belgium Department of Oral and Maxillofacial Surgery, North Manchester Hospital, Manchester, UK c Department of Oral and Maxillofacial Plastic Surgery, Rostock University, Schillingallee 35, 18057 Rostock, Germany b
a r t i c l e i n f o
a b s t r a c t
Article history: Paper received 23 February 2011 Accepted 29 March 2011
Introduction: Not many randomised controlled trials are published in surgical journals, especially those on maxillo-facial surgery. There appears to be some uncertainty on how to perform such studies. Accordingly this paper offers some information on how to plan, pursue and publish a well conducted case-control study, or the more powerful randomised control trial. Result: The main section describes how to define a relevant clinical question, and a research protocol, the way to implement the study, and it helps to find funding for such research. It also explains the various study designs, gives a very short introduction to statistics and on how to appraise the results achieved, and it advises on writing and submitting the resultant manuscript. Conclusion: This paper offers a guide for young colleagues who wish to perform a study, write a paper and achieve publication in one of our leading speciality journals. Ó 2011 European Association for Cranio-Maxillo-Facial Surgery.
Keywords: Randomised control trial Clinical study Evidence based therapy
1. Introduction Retrospective clinical studies are rarely free of bias. Their only advantage is that the series usually contains sufficient numbers for analysis at the time someone is interested in a clinical issue. However, data derived from clinical records lack precision and completeness. Clinical case series are descriptive and do not allow unbiased comparisons (no control group, whether operated upon by the “gold standard” technique or sham operated). That is why they represent weak evidence and are considered low in the pyramid of evidence (Table 1; Bellomo & Bagshaw, 2006). In order to develop surgical treatment standards, a prospective study is necessary with a large database. Normally no single maxillo-facial department will have sufficient patients with a specific problem or disease within an acceptable period, on which to base a study. Multicentre studies are good for enlarging the database more quickly. Often the development of a new surgical standard also needs the input of various specialities. These two ways of cooperation e more than one institution and more than one speciality e are better for shaping and testing a new therapy regime, more rapidly and perfectly than the usual method of defining, developing, and performing clinical research by just involving colleagues, patients, and facilities of just one institution.
* Corresponding author. E-mail address: [email protected] (K.K.H. Gundlach).
Prospective interdisciplinary and multicentre studies represent an ideal way to start clinical studies and to collect a good volume of relevant data e possibly even long term follow-up studies. However, a network of institutions co-operating in a patient-orientated clinical study needs a lot of methodological, technological, organisational, and electronic data information to meet all the requirements of co-operation between colleagues. Funding and language barriers have to be considered, as do fairness and quality control. In medicine, the term best evidence means the evidence furnished by well-designed studies (Sackett et al., 1996). It has been shown that high quality non-randomised (observational) studies and high quality Randomised Control Trials can produce similar answers and hence high quality (non-randomised) observational studies can be a good alternative to RCTs in surgery (Barton, 2000). Study estimates of effectiveness in observational studies may be valid if confounding factors are controlled (MacLehose et al., 2000), but formal agreement on the standards of non-randomised studies and meta-analyses of such studies are needed (Slim, 2005). This makes high quality RCTs and meta-analysis of high quality RCTs still the guarantee of best evidence (Table 1). Whilst Howes et al. (1997) stated that surgical practice was evidence based, Horton has demonstrated in 1996 that only 7% of studies published in leading surgical journals presented data derived from randomised controlled trials (Slim, 2005). This number did not increase over a 10-year time span (Wente et al., 2003; Panesar et al., 2006). Furthermore, in the main database for Evidence Based Surgery (i.e. the Cochrane Library), a search in February 2008 retrieved only 1 “hit” in the Journal of Cranio-
1010-5182/$ e see front matter Ó 2011 European Association for Cranio-Maxillo-Facial Surgery. doi:10.1016/j.jcms.2011.03.021
98
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
Table 1 A simplified and traditional hierarchy for grading the quality of evidence (modified after Urschel, 2005, and Bellomo & Bagshaw, 2006). Level Level Level Level
Ia Ib II III IIIa IIIb IIIc Level IV Level Va Vb
Well conducted, suitable powered RCT (randomised controlled trial) Meta-analyses and systematic reviews of multiple RCTs Well conducted, but small and under-powered RCT Non-randomised observational studies Prospective (concurrent) cohort studies Retrospective (historic) control studies Case-control studies Non-randomised study with historical controls Case series without controls Case reports
Maxillo-Facial Surgery for a systematic review and 3 for a clinical trial for all issues ever published. Finally, S. Brooke Steward et al. (2010) reported that among the 237 papers submitted for presentation at the 2009 Annual Scientific Conference of the British Association of Oral and Maxillofacial Surgeons there were 32 prospective/observational studies, 35 case reports/series, and 45 retrospective studies but only 1 case-control study and just 3 randomised trials. The paucity of meta-analyses in the surgical literature (due to the paucity of RCTs) remains indisputable. This does not mean that all interventions not based on RCTs are worthless! It has been estimated that even if barriers to randomisation could be overcome and more RCTs were performed, that 60% of surgical questions could not be answered by an RCT (Solomon & McLeod, 1995). In 1995, the proportion of surgical interventions based on RCT evidence was reported to be 24% (Ellis et al., 1995). However, based on apparently realistic assumptions, only 39% of surgical treatments could be subjected to RCT under ideal conditions (Solomon & McLeod, 1995). Whereas physicians can often simply dismiss case series from consideration, surgeons do not currently have this luxury. We must still critically analyse case series while at the same time encouraging more sophisticated research studies (Urschel, 2005). A number of factors may contribute to the difference with general medical journals, where five times more meta-analyses are published (Slim, 2005): 1. Funding of large multicentre surgical RCTs by national or commercial bodies is rare, and the knowledge that such funding is unlikely may discourage these. 2. Standardisation of a complex surgical procedure is a major challenge, unlike the standard administration of most drug regimens. 3. Performance between surgeons is likely to vary quite widely. Hence external validity may already be questionable. 4. Modification of the technique in response to particular circumstances in individual patients may reduce inclusions considerably (e.g. neck dissections). 5. Patients may be reluctant to participate as they may have a preference for a particular technique, e.g. “open sky” versus endoscopically assisted surgery. 6. The government imposes less stringent regulations on new operations and technologies than on new drugs. 7. Surgeons may have ethical problems with enrolling patients in a trial when they know they have to do a procedure with which they feel inexperienced. Surgeons are usually skilled in one operative approach to any given problem, but they are rarely equally proficient in two alternative operations. 8. Surgeons tend to be philosophically attached to their own surgical viewpoint or technique in a way that usually exceeds a physician’s affinity for a particular drug. This paper gives a step-by-step guide for colleagues who wish to start an evidence based clinical research project. It also explains how to answer the 4 questions put down by A.B. Hill in 1966: Why did you start? What did you do? What did you find? What does it mean anyway? (Evans, 2007).
2. Five essential steps (from original ideas to finished presentation) 2.1. 1st step: Defining a question There are three chief areas of study in cranio-maxillo-facial surgery: testing a surgical therapy, a (new) diagnostic parameter, or a (new) prognostic factor. The first task always is framing the relevant question. Is the issue actual, important and pertinent? Is the clinical situation such that there is doubt about effectiveness of a particular treatment? The next consideration is the need for in vitro or animal experimentation before embarking on a human experiment. Obviously, ethical considerations play an important role and are difficult to define. Prior to defining the clinical study idea in more detail, one has to find out whether this idea has already been studied by others, and if so how they have done it. Therefore a literature search is necessary. Helpful sources of information are Medline (www.ncbi.nlm.nih.gov/PubMed), Cochrane Library (www.cochrane.org) and Embase (www.embase. com). Once this is done, and assuming that the answer is not unequivocal from a properly conducted meta-analysis or randomised controlled trial (RCT), the study has to be defined in detail by posing the following four questions: Patients (P), Intervention/ Procedure/Factor (I), Comparison (C), Outcome (O); (“PICO”; Table 2; www.cebm.net/focus_quest.asp). When these four questions have been answered, the initially sound if vague idea will have developed into a well defined question: Is THIS NEW OPTION, thus leading to the DEFINED AIM, better than THE PREVIOUSLY ESTABLISHED METHOD? The next problem to be answered is ethical. The Ethics Committee of the hospital, university, state/region or country has to be contacted and the following three questions in particular need to be answered: Is there reason to believe that the benefits of the NEW OPTION are greater and/or less harmful than the PREVIOUS ONE? Is it necessary and if so is it ethical to randomise the patients to their treatment? Is it necessary and if so is it ethical to “blind” the patients1 and/or the health care provider from the input that has been given? Please note that all research on (human) patients has to comply with the principles laid down in the Declarations of Helsinki (1964) and their amendments (http://en.wikipedia.org/wiki/Declaration_ of_Helsinki).
2.2. 2nd step: Defining the protocol After defining the question, a hypothesis is formulated (this may be a “null” hypothesis) and an “antithesis”, a comparative procedure, is selected to test against it. A good idea is to first define the study outline (“Letter of Intent”) followed by a detailed protocol using the following checklist: a) Research Question (thesis and antithesis) and Definition of Aims. The aim must be clearly stated and should be clearly focused. Study goals should be formulated before data analysis. The exact procedure, intervention or factor has to be defined. b) Background and Significance of the Study. What is known already (see “literature search“) and why are the aims of this study important?
1 A trial is double blind if both, patients and research staff involved are kept unaware; it is single blind when only one of these two parties is unaware (most often the test subjects).
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
99
Table 2 Patients (P), Intervention/Procedure/Factor (I), Comparison (C), Outcome (O); (PICO; www.cebm.net/focus_quest.asp).
Patients Intervention, Procedure or Factor Comparison Outcome
Diagnostic
Therapeutic
Prognostic
What What What What
What What What What
What What What What
patients procedure(s) to be studied procedure(s) to compare with will be improved
c) Expected Outcome is to be defined including how to measure it. Accuracy and reliability of the measurement instrument(s) have to be determined. A valid measure is one that does what it says, an accurate measure is one that is very precise, and a reliable measure is one that gives a similar result when applied on more than one occasion. Remember, hospital records are not designed for research, and many measurements that are acceptable for clinical care are not valid enough for research. d) Methods to be used and Sketch of Research Plan. 1. Study design (see below). 2. Subject/material recruitment, sampling; inclusion and exclusion criteria. Demographic, clinical, geographic and temporal characteristics are to be defined. Examples for exclusion criteria are: a high likelihood of loss to follow-up, inability to provide quality data, higher risks of complications and characteristics that make it unethical to withhold a specific treatment. 3. So-called demographic predictors have to be identified that might influence the result (e.g. gender, cause(s) and type(s) of disorder, as well as outcome variables [e.g. radiographic/ laboratory findings, function, pain]). 4. Ethical considerations; handling safety, privacy, confidentiality; obtaining informed consent. 5. Statistical issues (see below). 6. Funding (see below, this actually being the third “essential step”). e) Time Frame, planned start and finish. Here it is again essential to contact a statistician who will indicate what numbers are needed to empower the study and achieve statistical “significance”. f) Quality Control, data management, compliance and follow-up. Where patients/materials will be recruited/measured and by whom; who is to collect primary data and manage it; who is to apply to the Ethics Committee(s) and where.
2.2.1. Re: Study design Reliability and validity can be tested in Diagnostic Studies. For these, most of this text is applicable but not everything is to be considered necessarily. For Therapeutic and Prognostic Studies there are various levels of quality of evidence (“degrees of significance”; McKibbon et al., 1999; Table 1). At the basic level, there are Case Reports and Case Series. Both are patient reports, either single or a group of these. They are descriptive in character, but as there are no control groups for comparison, neither has any statistical validity. However, unusual experience(s) may be noteworthy for peers. Next in the hierarchy of study designs are Case-Control Studies. They are one of the two types of observational studies (the other type is cohort studies). Case-control studies are retrospective, use patients who already have the disease or condition, and match an unaffected control group to see if the suspected differences exist. When suspected causal factors are found, then statistical testing is required to assist in defining real causality for that condition (D’Souza et al.,
patients intervention to be studied intervention to compare with will be improved
patients factor(s) to be studied other factors are known to influence outcome is a relevant outcome
2007). An example would be the proposed link between humanpapilloma-virus (HPV)-infection leading to oro-pharyngeal carcinoma. The great triumph for the case-control study was the demonstration of the link between tobacco smoking and lung cancer (Doll & Hill, 1950). Opponents, (usually backed by the tobacco industry) argued (correctly) for many years that this type of study cannot prove causation, but the eventual results of Double-blind Prospective Studies confirmed the causal link which the casecontrol studies had suggested. However, case-control studies are less reliable than cohort studies and randomised controlled trials. The other type of observational studies is the Cohort Study: This prospectively studies a group of matched individuals (the “cohort”) who differ only in the actual factor under scrutiny, to see how this influences any difference of outcome. For example, a group of 12 year old females is followed up to observe whether prescription of additional oestrogens influence mandibular growth. This is the better type of observational study and here a temporal relationship and outcome can be established. The disadvantage of this type of study, however, is that it may take a long time for the specific outcome to develop. However, it is a very good way to test for a prognostic factor. The apex of evidence is generally the well conducted and suitably powered (possibly multi-centre and multi-national) doubleblind placebo controlled randomised trial. A Randomised Controlled Trial (RCT) provides the strongest evidence for safety and effectiveness (McKibbon et al., 1999). For RCTs, a group of patients is collected in a prospective manner. These individuals are then assigned at random to either the experimental group (for the NEW test or treatment etc.) or to the control group which will receive the conventional/established test or treatment (or a placebo or sham operation). This type of test minimises the risk of confounding variables and offers the best known analysis for cause and effect. This basis then is much more solid than any other study design. However, ethical approval and informed consent are absolute requirements. Sometimes a blinded or even doubleblinded trial is not possible; and there always remains the question whether Placebo Surgery is unethical (Emanuel & Miller, 2002; Horng & Miller, 2002; Weijer, 2003). A Meta-Analysis is considered to be the top level of evidence, but it is prone to suffer from poor quality reporting and the inclusion of randomised trials of poor quality or including suboptimal data. 2.2.2. Re: Statistics At the very beginning of a prospective clinical study, it is essential that a statistician is involved. He/she will stipulate how large the study group needs to be (related to study-power), how to randomise, and how to analyse the data when the clinical trial is finished. 2.2.2.1. Alpha and beta errors, power of the study. An alpha or type I error describes the probability that a trial would find a positive result by chance for an intervention that seems to be effective when, in fact, it is not (“false positive”). The probability of an alpha error for any given trial in medicine is traditionally limited to (or set at) <0,05. Recent trends have brought greater recognition for hypothesis testing by use of confidence intervals (CI). However, to
100
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
consider a type I error still remains frequent for statistical purposes and sample size estimation in trial design. Indeed, the possibility of an alpha error is generally inversely related to the study sample size. Level I evidence demands that trials should have a low probability of committing an alpha error. The probability of an alpha error to occur, however, is clinically or statistically difficult to measure beforehand. But, the level of confidence has to be chosen prior to starting a study and not after evaluating the test results. A “tendency” is a poor term and merely indicates that possibly the number of tested patients/materials was too small. And “highly significant” is bragging (And differentiating between one and up to three stars for quality/statistical significance should be reserved for qualifying cognac! Ulmer, 2010). A beta or type II error describes a statistical error where a trial would find that an intervention is negative (i.e. not effective) when, in fact, it is not (“false negative”). A larger study sample size reduces the probability of a beta error on the assumption that a genuine difference in effect exists between intervention groups. The probability of beta error is traditionally set at 0,10e0,20 (“power” 0,80e0,90). The risk of a beta error can be reduced by making rational assumptions, based on available evidence, about the likelihood of a given outcome being observed in the control arm of the trial, and on the size of treatment effect. Maximising (trial) power may seem logical, but also has ethical and cost considerations (Bellomo & Bagshaw, 2006). 2.2.2.2. Descriptive and analytical statistics. Firstly, Descriptive Statistics are used to compare all patients enrolled in the study and control groups in order to avoid any selection bias or “confounding”. This is also used to define the extent of loss to follow-up. Here the values of arithmetic mean, median, range, and standard deviation are used. These also enable determination of the range and the field of application of the study results once it has been finished and evaluated (related to external validity). Next are Analytical Statistics, which help to distinguish results achieved by chance from true differences or associations. For clinical studies, the most frequently used analyses are: 1) Student’s t-test (for quantitative data). There are the “normal” or routine t-test (to compare un-paired arithmetic means from two independent samples) and the other one, the “paired” t-test (when two dependent samples are compared with each other). 2) ANalysis Of VAriance (ANOVA) is the test used when more than two groups are involved and quantitative data have to be evaluated. ANOVA can tell whether there are statistical differences of means between the groups (the F-ratio). It can also tell which group differs from the other ones (Scheffé test), and it can also tell whether these differences are relatively small or great. 3) The Chi-square test is applied to qualitative data, i.e. when nominal or categorical data (e.g. success vs. failure, big vs. small, green vs. red, etc.) are compared regarding their frequencies. There are the Chi-square “goodness of fit” test (to test any theory regarding the shape of a population) and the Chi-square “test of independence” (to test whether there is an association between the frequencies or proportions of one category against the other category [e.g. age and blood pressure]). 4) Pearson’s Correlation and Regression. When calculating “r”, the coefficient of correlation, one may find whether there is a correlation, i.e. some type of association between two variables. An r-value of 0 specifies that there is no relationship between the two. Also, when calculating a regression line in order to find out the “best” correlation (or line of fit between two variables on a scatter graph) one is only demonstrating that there is some kind of relationship between the two, but not which one of the two is causing the other.
2.3. 3rd step: Funding Another problem is obtaining funding. The best funding sources are: a) Governmental agencies of the country in question. In Germany for example, there is the Ministry for Scientific Research (BMBF, Bundesministerium für Bildung und Forschung) which e for the time being e is especially helpful with multicentre pharmacological studies. The National Fund for Scientific Research (NFSR; in Dutch: NFW, in French: FNRS) is the best known Institution in Belgium for supporting scientific research, and in Britain it is the Medical Research Council. b) Charitable, not-for-profit Institutions. In Germany for example there is the German Society Research Foundation (DFG, Deutsche Forschungs-Gemeinschaft) which supports non-pharmacological multicentre or supra-regional studies dealing with ideas regarding new diagnostic as well as therapeutic (including prognostic or preventive) measures (It also contributes money for acquiring material or hiring personnel necessary for single centre studies). c) There are also grants from the European Union. These can be applied for by clicking on www.europa.eu, then on “quick link for schools & universities”, then “research”, then “links for researchers” and then for example “(European Commission) Cordis” and then finally “funding”. There are several annual work programmes one of these being “Health”, always inviting calls for specific topics. Clinical research is especially sought after, e.g. “translating the results of research outcomes into clinical practice, including better use of medicines, and appropriate use of behavioural and organisational interventions and health therapies and technologies”. The EU also supports “Ideas” to be investigated first by singular researchers, and supports “People” (single persons or networks e “Marie Curie”) to travel to other institutions in order to improve collaboration in studies or acquire special knowledge in techniques. The Seventh Framework Programme (FP7, http://cordis.europa. eu/fp7/home_en.html) bundles all research-related EU initiatives together under a common roof playing a crucial role in reaching the goals of growth, competitiveness and employment. It is also a key pillar for the European Research Area (ERA). For these three types of sources it is a prerequisite to have performed some research already related to the topic in question, and to have published the results thereof in accredited journals. d) However, there are two more financial possibilities to be considered, namely intramural support and industry sponsors. 1. Intramural support: Most universities will help young staff members financially in order to develop their interest in research and to finance their first steps to develop a topic for research. 2. Lastly industry sponsors are always a potentially funding source. However, a high quality journal will always ask their authors whether there is any conflict of interest. This means that any financial or personal relationship with companies, organisations (or other people) has to be declared in order to identify and exclude any possible bias.
2.4. 4th step: Implementing the study plan After deciding upon the type of study and defining the protocol, a Study Operations Manual providing detailed instructions for every single study procedure has to be compiled. This should
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
include the step-by-step process for enrolling and following patients, entering and managing data, and monitoring the process. Copies of all study materials, including study protocol, consent forms (in all the languages used), questionnaires, etc. should also be included in the manual. Procedures for maintaining confidentiality and quality assurance and control should also be covered. Here one should always consider and incorporate all relevant and applicable facts from the so-called Good Clinical Practice (GCP) procedures. This is an international ethical and scientific quality standard for designing, conducting, recording, and reporting trials that involve the participation of human subjects (www.emea.eu.int/pdfs/human/ich/013595en.pdf). When implementing the study plan it is important to filter the IMPORTANT aspects of the GCP-guidelines and to present them in a USER FRIENDLY format. 2.5. 5th step: Appraisal of results, report writing and submission for publication a) Appraisal of results Avoid data dredging and best test seeking. With data dredging, multiple possible associations in the data are tested in the hope of finding something significant. This gives rise to spurious associations. With best test seeking, the investigator seeks out good tests instead of good associations, which tends to overstate the statistical significance of an association. The basic data have to be properly described (age, sex, deformity/disease, type of practice/investigation set-up). This is required to generalise the study findings to the surgical practice of the reader (external validity). The measures of effect and the statistical significance have to be properly presented. The benefit offered by the object of the study, THE NEW OPTION, e.g. the risk reduction should be clearly presented. “Absolute” risk reduction is the risk in the control group minus the risk in the treatment group. “Relative” risk reduction is the absolute risk reduction divided by the risk in the control group, expressed as percentage.
101
One should focus on the main finding and not emphasize a finding in a subgroup, when this was not planned for a priori. Try to eliminate any bias and to understand any confounders. Bias can systematically deviate the results away from the truth. Confounders are variables that are responsible for an apparent, but false, association between two study variables. b) Report writing and submission for publication First of all one has to decide which audience is to be addressed. This will then lead to the journals that are most appropriate. In order to choose which of these, you might visit the website of the journals and look at their tables of content in order to find out about their mission and interest. It may also be of interest to see how long the intervals are between submission, acceptance, and publication in these journals. It may be important to learn about the current impact factor of the journals in question. The editorin-chief may be asked for this information (e-mail address on publisher’s website). Once the journal has been decided upon, one should always inquire about its guidelines of formatting and submission. So-called “instructions for authors” are to be found in printed issues of that journal. When writing for scientific publication, as a rule, one should always start with an Introduction that informs the reader about the relevant area and the problem this paper is tackling, the reason why this is an important problem, and how the investigation was performed. The Material and Methods section should describe the type of study (e.g. cohort study), all the inclusion and exclusion criteria, the period during which the data has been collected, the institutions involved, the descriptive analytical data of the patients/material collected should be described (including age, sex, recruitment, enrolment, ways of application of the various treatments, loss to follow-up, definition of outcome criteria, and how determined or measured, analysed, and compiled). This should be supplemented by the statistical tests applied e and the reliability and the precision of data measurement, collection, and analysis.
Table 3 Reporting quality checklist (based on the CONSORT checklist; after Moher et al., 2001). Paper selection & topic
CONSORT item number
Descriptor
Title & Abstract Methods Participants
1
The study should be identifiable as a RCT.
3
Intervention
4
Objectives Outcome
5 6
The planned study population should be defined. Inclusion and exclusion criteria should be defined. Each intervention should be fully described, including name of products, name of producer, city, and country. An aim or (null) hypothesis should be stated. Primary and secondary outcome measures should be defined. Measurement errors should be reported. How was the sample size determined? The method used to generate the random allocation sequence should be stated. The method used to implement the random allocation sequence should be stated. How was allocation sequence concealed from executor of assignment? How were patients blinded to group assignment? Were the investigator and/or assessor blinded to the assignment? The statistical methods used to compare groups for primary outcome should be stated.
Sample size Sequence generation Allocation concealment Implementation Blinded participants Blinded investigator/assessor Statistical methods Results Participant flow Baseline data Number analysed Adverse effects Discussion Interpretation
7 8 9 10 11a 11b 12
13 15 16 19 20
A flow diagram or description of non-compliance/drop out has to be provided/is very helpful Demographic data have to be included Was the analysis based on “intention-to-treat”? All complications should be described Limitations such as bias, possible confounding factors and imprecision should be discussed
102
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
Quality of reporting is very important when dealing with RCTs. Sample size calculation, randomisation methodology, allocation concealment and blind investigation/assessment are often not adequately reported or are omitted (Schulz et al., 1995; Karri, 2006). In the Results section all the results obtained including the descriptive and the analytical statistical results should be listed. Here tables will often help. Raw data should be kept available for an interested reader, and this can be in different formats (table, attachment, or available upon request). And in the Discussion the results are compared with those reported by others in the scientific literature. The relevancies of statistical and of clinical significances should be discussed. Credibility and logical connections should also be checked: “Does it make sense?” One should avoid only citing supportive data. Finally a paragraph on the implications of the results obtained should follow, together with indications for generalisation and for future research in the area. Honour requires that the weaknesses and limitations of the study are pointed out before ending with a short and concise conclusion. The quality of reporting can be improved by following the CONSORT statement (Moher et al., 2001; Karri, 2006; Table 3). The Conclusion is the last section with a short summary of the most important findings and their inference for diagnosing or treating the disorder in question. Here, any speculation should be avoided. Linguistic corrections: Authors whose mother tongue is not English often underestimate the importance of correct English in the manuscript, and think that the technical editor will take care of it. In Europe, most of the reviewers are also of non-English origin and understanding the text is the first prerequisite for judging the quality of the content. The quality of the language and of the formatting unfortunately do play a role in deciding which attitude a reviewer unconsciously, or consciously has towards a manuscript. Companies specialised in editing, writing and proofreading can be of help (sometimes even great help), although often at a significant cost. When the paper has been returned to the authors from the Editor-in-Chief, there will almost never be a prompt acceptance of the paper in the format it was submitted. There will most often be some kind of criticism. This should be interpreted as a helpful way of improving the paper. Apart from linguistic shortcomings, comments involve mainly the scientific basis of the interpretation of data in the discussion. If the criticism is not justified or even unfair, it is alright to answer with a letter of polite dissent disputing the editorial board members’ opinions. If it was rejected the manuscript should not just be dropped. Rather it should be corrected, improved and then either resubmitted or sent to another journal. Perseverance and patience are required. Of the 97 manuscripts reviewed for the journal “Plastic and Reconstructive Surgery” by one reviewer between 1992 and 2003, not one was regarded to be “acceptable as is”, of 44 the reviewer felt that “major revision was needed”, only 44 were published in the targeted journal, whilst 22 were published in a PubMed-indexed journal other than the one targeted (Loonen et al., 2005). 3. Conclusion This paper is a guide for young residents who want to plan, perform and publish a study on their new clinical research in a peer
reviewed journal. The main emphasis is laid on the thoughts and techniques necessary for designing, pursuing and writing of a randomised control trial. But this article may also help in producing other kinds of manuscripts, e.g. for preparing a clinical observational study. Conflict of interests notification There is no other interest or conflict of interests to be disclosed for anyone of the three authors. List of sponsors No grants were received for this scientific paper and no sponsor was supporting it either. References Barton S: Which clinical studies provide the best evidence? Br Med J 321: 255e256, 2000 Bellomo R, Bagshaw SM: Evidence-based medicine: classifying the evidence from clinical trials - the need to consider other dimensions. Crit Care 10: 1-8, 2006 (available online: http://ccforum.com/content/10/5/232) Brooke Steward S, Oeppen RS, Cascarini L, Brennan PA: Educational article: what gets accepted for presentation? e A study of submitted abstracts for the 2009 BAOMS conference. Br J Oral Maxillofac Surg 48: 297e300, 2010 Doll R, Hill AB: Smoking and carcinoma of the lung. Br Med J 2(4682): 739e748, 1950 D’Souza G, Kreimer AR, Viscidi R, Pawlita ML, Fakhry C, Koch WM, et al: Casecontrol study of human papilloma virus and oropharyngeal cancer. N Engl J Med 356: 1944e1956, 2007 Ellis J, Mulligan J, Rowe J, Sackett DL: Inpatient general medicine is evidence based. A-Team, Nuffield Department of Clinical Medicine. Lancet 346: 407e410, 1995 Emanuel EJ, Miller FG: The ethics of placebo-controlled trials. N Engl J Med 346: 382e383, 2002 Evans M: Writing a paper. Br J Oral Maxillofac Surg 45: 485e487, 2007 Horng S, Miller FG: Is placebo surgery unethical? N Engl J Med 347: 137e139, 2002 Horton R: Surgical research or comic opera: questions, but few answers. Lancet 347: 984e985, 1996 Howes N, Chagla L, Thorpe M, McCulloch P: Surgical practice is evidence based. Br J Surg 84: 1220e1223, 1997 Karri V: Randomised clinical trials in plastic surgery: survey of output and quality of reporting. J Plast Reconstr Aesthet Surg 59: 787e796, 2006 Loonen MPJ, Hage JJ, Kon M: Who benefits from peer review? An analysis of the outcome of 100 requests for review by plastic and reconstructive surgery. Plast Reconstr Surg 116: 1461e1472, 2005 MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM: A systematic review of comparisons of effect sizes derived from randomised and nonrandomised studies. Health Technol Assess 4: 1e154, 2000 McKibbon A, Eady A, Marks S: PDQ-evidence-based principles and practice. Hamilton, Ontario (Canada): BC Decker, 1999 Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357: 1191e1194, 2001 Panesar SS, Thakrar R, Athanasiou T, Sheikh A: Comparison of reports of randomised controlled trials and systematic reviews in surgical journals: literature review. J R Soc Med 99: 470e472, 2006 Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. Br Med J 312: 71e72, 1996 Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408e412, 1995 Slim K: Limits of evidence-based surgery. World J Surg 29: 606e609, 2005 Solomon MJ, McLeod RS: Should we be performing more randomized controlled trials evaluating surgical operations? Surgery 118: 459e467, 1995 Ulmer H-V: Differenz erfragen. In Diskussion zu. In: Victor A, Elsässer A, Hommel G, Blettner M (eds) Wie bewertet man die p-Wert-Flut? Dtsch Arztebl, vol. 107; 2010, 417, 2010 Urschel JD: How to analyze an article. World J Surg 29: 557e560, 2005 Weijer C: The ethics of placebo-controlled trials. J Bone Miner Res 18: 1150e1153, 2003 Wente MN, Seiler CM, Uhl W, Büchler MW: Perspectives of evidence-based surgery. Dig Surg 20, Epub 2003.
Contents lists available at ScienceDirect
Journal of Cranio-Maxillo-Facial Surgery journal homepage: www.jcmfs.com
How to do clinical research in cranio-maxillo-facial surgery Maurice Y. Mommaerts a, Murray E. Foster b, Karsten K.H. Gundlach c, * a
Division of Maxillo-Facial Surgery, AZ St. Jan, Bruges, Belgium Department of Oral and Maxillofacial Surgery, North Manchester Hospital, Manchester, UK c Department of Oral and Maxillofacial Plastic Surgery, Rostock University, Schillingallee 35, 18057 Rostock, Germany b
a r t i c l e i n f o
a b s t r a c t
Article history: Paper received 23 February 2011 Accepted 29 March 2011
Introduction: Not many randomised controlled trials are published in surgical journals, especially those on maxillo-facial surgery. There appears to be some uncertainty on how to perform such studies. Accordingly this paper offers some information on how to plan, pursue and publish a well conducted case-control study, or the more powerful randomised control trial. Result: The main section describes how to define a relevant clinical question, and a research protocol, the way to implement the study, and it helps to find funding for such research. It also explains the various study designs, gives a very short introduction to statistics and on how to appraise the results achieved, and it advises on writing and submitting the resultant manuscript. Conclusion: This paper offers a guide for young colleagues who wish to perform a study, write a paper and achieve publication in one of our leading speciality journals. Ó 2011 European Association for Cranio-Maxillo-Facial Surgery.
Keywords: Randomised control trial Clinical study Evidence based therapy
1. Introduction Retrospective clinical studies are rarely free of bias. Their only advantage is that the series usually contains sufficient numbers for analysis at the time someone is interested in a clinical issue. However, data derived from clinical records lack precision and completeness. Clinical case series are descriptive and do not allow unbiased comparisons (no control group, whether operated upon by the “gold standard” technique or sham operated). That is why they represent weak evidence and are considered low in the pyramid of evidence (Table 1; Bellomo & Bagshaw, 2006). In order to develop surgical treatment standards, a prospective study is necessary with a large database. Normally no single maxillo-facial department will have sufficient patients with a specific problem or disease within an acceptable period, on which to base a study. Multicentre studies are good for enlarging the database more quickly. Often the development of a new surgical standard also needs the input of various specialities. These two ways of cooperation e more than one institution and more than one speciality e are better for shaping and testing a new therapy regime, more rapidly and perfectly than the usual method of defining, developing, and performing clinical research by just involving colleagues, patients, and facilities of just one institution.
* Corresponding author. E-mail address: [email protected] (K.K.H. Gundlach).
Prospective interdisciplinary and multicentre studies represent an ideal way to start clinical studies and to collect a good volume of relevant data e possibly even long term follow-up studies. However, a network of institutions co-operating in a patient-orientated clinical study needs a lot of methodological, technological, organisational, and electronic data information to meet all the requirements of co-operation between colleagues. Funding and language barriers have to be considered, as do fairness and quality control. In medicine, the term best evidence means the evidence furnished by well-designed studies (Sackett et al., 1996). It has been shown that high quality non-randomised (observational) studies and high quality Randomised Control Trials can produce similar answers and hence high quality (non-randomised) observational studies can be a good alternative to RCTs in surgery (Barton, 2000). Study estimates of effectiveness in observational studies may be valid if confounding factors are controlled (MacLehose et al., 2000), but formal agreement on the standards of non-randomised studies and meta-analyses of such studies are needed (Slim, 2005). This makes high quality RCTs and meta-analysis of high quality RCTs still the guarantee of best evidence (Table 1). Whilst Howes et al. (1997) stated that surgical practice was evidence based, Horton has demonstrated in 1996 that only 7% of studies published in leading surgical journals presented data derived from randomised controlled trials (Slim, 2005). This number did not increase over a 10-year time span (Wente et al., 2003; Panesar et al., 2006). Furthermore, in the main database for Evidence Based Surgery (i.e. the Cochrane Library), a search in February 2008 retrieved only 1 “hit” in the Journal of Cranio-
1010-5182/$ e see front matter Ó 2011 European Association for Cranio-Maxillo-Facial Surgery. doi:10.1016/j.jcms.2011.03.021
98
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
Table 1 A simplified and traditional hierarchy for grading the quality of evidence (modified after Urschel, 2005, and Bellomo & Bagshaw, 2006). Level Level Level Level
Ia Ib II III IIIa IIIb IIIc Level IV Level Va Vb
Well conducted, suitable powered RCT (randomised controlled trial) Meta-analyses and systematic reviews of multiple RCTs Well conducted, but small and under-powered RCT Non-randomised observational studies Prospective (concurrent) cohort studies Retrospective (historic) control studies Case-control studies Non-randomised study with historical controls Case series without controls Case reports
Maxillo-Facial Surgery for a systematic review and 3 for a clinical trial for all issues ever published. Finally, S. Brooke Steward et al. (2010) reported that among the 237 papers submitted for presentation at the 2009 Annual Scientific Conference of the British Association of Oral and Maxillofacial Surgeons there were 32 prospective/observational studies, 35 case reports/series, and 45 retrospective studies but only 1 case-control study and just 3 randomised trials. The paucity of meta-analyses in the surgical literature (due to the paucity of RCTs) remains indisputable. This does not mean that all interventions not based on RCTs are worthless! It has been estimated that even if barriers to randomisation could be overcome and more RCTs were performed, that 60% of surgical questions could not be answered by an RCT (Solomon & McLeod, 1995). In 1995, the proportion of surgical interventions based on RCT evidence was reported to be 24% (Ellis et al., 1995). However, based on apparently realistic assumptions, only 39% of surgical treatments could be subjected to RCT under ideal conditions (Solomon & McLeod, 1995). Whereas physicians can often simply dismiss case series from consideration, surgeons do not currently have this luxury. We must still critically analyse case series while at the same time encouraging more sophisticated research studies (Urschel, 2005). A number of factors may contribute to the difference with general medical journals, where five times more meta-analyses are published (Slim, 2005): 1. Funding of large multicentre surgical RCTs by national or commercial bodies is rare, and the knowledge that such funding is unlikely may discourage these. 2. Standardisation of a complex surgical procedure is a major challenge, unlike the standard administration of most drug regimens. 3. Performance between surgeons is likely to vary quite widely. Hence external validity may already be questionable. 4. Modification of the technique in response to particular circumstances in individual patients may reduce inclusions considerably (e.g. neck dissections). 5. Patients may be reluctant to participate as they may have a preference for a particular technique, e.g. “open sky” versus endoscopically assisted surgery. 6. The government imposes less stringent regulations on new operations and technologies than on new drugs. 7. Surgeons may have ethical problems with enrolling patients in a trial when they know they have to do a procedure with which they feel inexperienced. Surgeons are usually skilled in one operative approach to any given problem, but they are rarely equally proficient in two alternative operations. 8. Surgeons tend to be philosophically attached to their own surgical viewpoint or technique in a way that usually exceeds a physician’s affinity for a particular drug. This paper gives a step-by-step guide for colleagues who wish to start an evidence based clinical research project. It also explains how to answer the 4 questions put down by A.B. Hill in 1966: Why did you start? What did you do? What did you find? What does it mean anyway? (Evans, 2007).
2. Five essential steps (from original ideas to finished presentation) 2.1. 1st step: Defining a question There are three chief areas of study in cranio-maxillo-facial surgery: testing a surgical therapy, a (new) diagnostic parameter, or a (new) prognostic factor. The first task always is framing the relevant question. Is the issue actual, important and pertinent? Is the clinical situation such that there is doubt about effectiveness of a particular treatment? The next consideration is the need for in vitro or animal experimentation before embarking on a human experiment. Obviously, ethical considerations play an important role and are difficult to define. Prior to defining the clinical study idea in more detail, one has to find out whether this idea has already been studied by others, and if so how they have done it. Therefore a literature search is necessary. Helpful sources of information are Medline (www.ncbi.nlm.nih.gov/PubMed), Cochrane Library (www.cochrane.org) and Embase (www.embase. com). Once this is done, and assuming that the answer is not unequivocal from a properly conducted meta-analysis or randomised controlled trial (RCT), the study has to be defined in detail by posing the following four questions: Patients (P), Intervention/ Procedure/Factor (I), Comparison (C), Outcome (O); (“PICO”; Table 2; www.cebm.net/focus_quest.asp). When these four questions have been answered, the initially sound if vague idea will have developed into a well defined question: Is THIS NEW OPTION, thus leading to the DEFINED AIM, better than THE PREVIOUSLY ESTABLISHED METHOD? The next problem to be answered is ethical. The Ethics Committee of the hospital, university, state/region or country has to be contacted and the following three questions in particular need to be answered: Is there reason to believe that the benefits of the NEW OPTION are greater and/or less harmful than the PREVIOUS ONE? Is it necessary and if so is it ethical to randomise the patients to their treatment? Is it necessary and if so is it ethical to “blind” the patients1 and/or the health care provider from the input that has been given? Please note that all research on (human) patients has to comply with the principles laid down in the Declarations of Helsinki (1964) and their amendments (http://en.wikipedia.org/wiki/Declaration_ of_Helsinki).
2.2. 2nd step: Defining the protocol After defining the question, a hypothesis is formulated (this may be a “null” hypothesis) and an “antithesis”, a comparative procedure, is selected to test against it. A good idea is to first define the study outline (“Letter of Intent”) followed by a detailed protocol using the following checklist: a) Research Question (thesis and antithesis) and Definition of Aims. The aim must be clearly stated and should be clearly focused. Study goals should be formulated before data analysis. The exact procedure, intervention or factor has to be defined. b) Background and Significance of the Study. What is known already (see “literature search“) and why are the aims of this study important?
1 A trial is double blind if both, patients and research staff involved are kept unaware; it is single blind when only one of these two parties is unaware (most often the test subjects).
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
99
Table 2 Patients (P), Intervention/Procedure/Factor (I), Comparison (C), Outcome (O); (PICO; www.cebm.net/focus_quest.asp).
Patients Intervention, Procedure or Factor Comparison Outcome
Diagnostic
Therapeutic
Prognostic
What What What What
What What What What
What What What What
patients procedure(s) to be studied procedure(s) to compare with will be improved
c) Expected Outcome is to be defined including how to measure it. Accuracy and reliability of the measurement instrument(s) have to be determined. A valid measure is one that does what it says, an accurate measure is one that is very precise, and a reliable measure is one that gives a similar result when applied on more than one occasion. Remember, hospital records are not designed for research, and many measurements that are acceptable for clinical care are not valid enough for research. d) Methods to be used and Sketch of Research Plan. 1. Study design (see below). 2. Subject/material recruitment, sampling; inclusion and exclusion criteria. Demographic, clinical, geographic and temporal characteristics are to be defined. Examples for exclusion criteria are: a high likelihood of loss to follow-up, inability to provide quality data, higher risks of complications and characteristics that make it unethical to withhold a specific treatment. 3. So-called demographic predictors have to be identified that might influence the result (e.g. gender, cause(s) and type(s) of disorder, as well as outcome variables [e.g. radiographic/ laboratory findings, function, pain]). 4. Ethical considerations; handling safety, privacy, confidentiality; obtaining informed consent. 5. Statistical issues (see below). 6. Funding (see below, this actually being the third “essential step”). e) Time Frame, planned start and finish. Here it is again essential to contact a statistician who will indicate what numbers are needed to empower the study and achieve statistical “significance”. f) Quality Control, data management, compliance and follow-up. Where patients/materials will be recruited/measured and by whom; who is to collect primary data and manage it; who is to apply to the Ethics Committee(s) and where.
2.2.1. Re: Study design Reliability and validity can be tested in Diagnostic Studies. For these, most of this text is applicable but not everything is to be considered necessarily. For Therapeutic and Prognostic Studies there are various levels of quality of evidence (“degrees of significance”; McKibbon et al., 1999; Table 1). At the basic level, there are Case Reports and Case Series. Both are patient reports, either single or a group of these. They are descriptive in character, but as there are no control groups for comparison, neither has any statistical validity. However, unusual experience(s) may be noteworthy for peers. Next in the hierarchy of study designs are Case-Control Studies. They are one of the two types of observational studies (the other type is cohort studies). Case-control studies are retrospective, use patients who already have the disease or condition, and match an unaffected control group to see if the suspected differences exist. When suspected causal factors are found, then statistical testing is required to assist in defining real causality for that condition (D’Souza et al.,
patients intervention to be studied intervention to compare with will be improved
patients factor(s) to be studied other factors are known to influence outcome is a relevant outcome
2007). An example would be the proposed link between humanpapilloma-virus (HPV)-infection leading to oro-pharyngeal carcinoma. The great triumph for the case-control study was the demonstration of the link between tobacco smoking and lung cancer (Doll & Hill, 1950). Opponents, (usually backed by the tobacco industry) argued (correctly) for many years that this type of study cannot prove causation, but the eventual results of Double-blind Prospective Studies confirmed the causal link which the casecontrol studies had suggested. However, case-control studies are less reliable than cohort studies and randomised controlled trials. The other type of observational studies is the Cohort Study: This prospectively studies a group of matched individuals (the “cohort”) who differ only in the actual factor under scrutiny, to see how this influences any difference of outcome. For example, a group of 12 year old females is followed up to observe whether prescription of additional oestrogens influence mandibular growth. This is the better type of observational study and here a temporal relationship and outcome can be established. The disadvantage of this type of study, however, is that it may take a long time for the specific outcome to develop. However, it is a very good way to test for a prognostic factor. The apex of evidence is generally the well conducted and suitably powered (possibly multi-centre and multi-national) doubleblind placebo controlled randomised trial. A Randomised Controlled Trial (RCT) provides the strongest evidence for safety and effectiveness (McKibbon et al., 1999). For RCTs, a group of patients is collected in a prospective manner. These individuals are then assigned at random to either the experimental group (for the NEW test or treatment etc.) or to the control group which will receive the conventional/established test or treatment (or a placebo or sham operation). This type of test minimises the risk of confounding variables and offers the best known analysis for cause and effect. This basis then is much more solid than any other study design. However, ethical approval and informed consent are absolute requirements. Sometimes a blinded or even doubleblinded trial is not possible; and there always remains the question whether Placebo Surgery is unethical (Emanuel & Miller, 2002; Horng & Miller, 2002; Weijer, 2003). A Meta-Analysis is considered to be the top level of evidence, but it is prone to suffer from poor quality reporting and the inclusion of randomised trials of poor quality or including suboptimal data. 2.2.2. Re: Statistics At the very beginning of a prospective clinical study, it is essential that a statistician is involved. He/she will stipulate how large the study group needs to be (related to study-power), how to randomise, and how to analyse the data when the clinical trial is finished. 2.2.2.1. Alpha and beta errors, power of the study. An alpha or type I error describes the probability that a trial would find a positive result by chance for an intervention that seems to be effective when, in fact, it is not (“false positive”). The probability of an alpha error for any given trial in medicine is traditionally limited to (or set at) <0,05. Recent trends have brought greater recognition for hypothesis testing by use of confidence intervals (CI). However, to
100
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
consider a type I error still remains frequent for statistical purposes and sample size estimation in trial design. Indeed, the possibility of an alpha error is generally inversely related to the study sample size. Level I evidence demands that trials should have a low probability of committing an alpha error. The probability of an alpha error to occur, however, is clinically or statistically difficult to measure beforehand. But, the level of confidence has to be chosen prior to starting a study and not after evaluating the test results. A “tendency” is a poor term and merely indicates that possibly the number of tested patients/materials was too small. And “highly significant” is bragging (And differentiating between one and up to three stars for quality/statistical significance should be reserved for qualifying cognac! Ulmer, 2010). A beta or type II error describes a statistical error where a trial would find that an intervention is negative (i.e. not effective) when, in fact, it is not (“false negative”). A larger study sample size reduces the probability of a beta error on the assumption that a genuine difference in effect exists between intervention groups. The probability of beta error is traditionally set at 0,10e0,20 (“power” 0,80e0,90). The risk of a beta error can be reduced by making rational assumptions, based on available evidence, about the likelihood of a given outcome being observed in the control arm of the trial, and on the size of treatment effect. Maximising (trial) power may seem logical, but also has ethical and cost considerations (Bellomo & Bagshaw, 2006). 2.2.2.2. Descriptive and analytical statistics. Firstly, Descriptive Statistics are used to compare all patients enrolled in the study and control groups in order to avoid any selection bias or “confounding”. This is also used to define the extent of loss to follow-up. Here the values of arithmetic mean, median, range, and standard deviation are used. These also enable determination of the range and the field of application of the study results once it has been finished and evaluated (related to external validity). Next are Analytical Statistics, which help to distinguish results achieved by chance from true differences or associations. For clinical studies, the most frequently used analyses are: 1) Student’s t-test (for quantitative data). There are the “normal” or routine t-test (to compare un-paired arithmetic means from two independent samples) and the other one, the “paired” t-test (when two dependent samples are compared with each other). 2) ANalysis Of VAriance (ANOVA) is the test used when more than two groups are involved and quantitative data have to be evaluated. ANOVA can tell whether there are statistical differences of means between the groups (the F-ratio). It can also tell which group differs from the other ones (Scheffé test), and it can also tell whether these differences are relatively small or great. 3) The Chi-square test is applied to qualitative data, i.e. when nominal or categorical data (e.g. success vs. failure, big vs. small, green vs. red, etc.) are compared regarding their frequencies. There are the Chi-square “goodness of fit” test (to test any theory regarding the shape of a population) and the Chi-square “test of independence” (to test whether there is an association between the frequencies or proportions of one category against the other category [e.g. age and blood pressure]). 4) Pearson’s Correlation and Regression. When calculating “r”, the coefficient of correlation, one may find whether there is a correlation, i.e. some type of association between two variables. An r-value of 0 specifies that there is no relationship between the two. Also, when calculating a regression line in order to find out the “best” correlation (or line of fit between two variables on a scatter graph) one is only demonstrating that there is some kind of relationship between the two, but not which one of the two is causing the other.
2.3. 3rd step: Funding Another problem is obtaining funding. The best funding sources are: a) Governmental agencies of the country in question. In Germany for example, there is the Ministry for Scientific Research (BMBF, Bundesministerium für Bildung und Forschung) which e for the time being e is especially helpful with multicentre pharmacological studies. The National Fund for Scientific Research (NFSR; in Dutch: NFW, in French: FNRS) is the best known Institution in Belgium for supporting scientific research, and in Britain it is the Medical Research Council. b) Charitable, not-for-profit Institutions. In Germany for example there is the German Society Research Foundation (DFG, Deutsche Forschungs-Gemeinschaft) which supports non-pharmacological multicentre or supra-regional studies dealing with ideas regarding new diagnostic as well as therapeutic (including prognostic or preventive) measures (It also contributes money for acquiring material or hiring personnel necessary for single centre studies). c) There are also grants from the European Union. These can be applied for by clicking on www.europa.eu, then on “quick link for schools & universities”, then “research”, then “links for researchers” and then for example “(European Commission) Cordis” and then finally “funding”. There are several annual work programmes one of these being “Health”, always inviting calls for specific topics. Clinical research is especially sought after, e.g. “translating the results of research outcomes into clinical practice, including better use of medicines, and appropriate use of behavioural and organisational interventions and health therapies and technologies”. The EU also supports “Ideas” to be investigated first by singular researchers, and supports “People” (single persons or networks e “Marie Curie”) to travel to other institutions in order to improve collaboration in studies or acquire special knowledge in techniques. The Seventh Framework Programme (FP7, http://cordis.europa. eu/fp7/home_en.html) bundles all research-related EU initiatives together under a common roof playing a crucial role in reaching the goals of growth, competitiveness and employment. It is also a key pillar for the European Research Area (ERA). For these three types of sources it is a prerequisite to have performed some research already related to the topic in question, and to have published the results thereof in accredited journals. d) However, there are two more financial possibilities to be considered, namely intramural support and industry sponsors. 1. Intramural support: Most universities will help young staff members financially in order to develop their interest in research and to finance their first steps to develop a topic for research. 2. Lastly industry sponsors are always a potentially funding source. However, a high quality journal will always ask their authors whether there is any conflict of interest. This means that any financial or personal relationship with companies, organisations (or other people) has to be declared in order to identify and exclude any possible bias.
2.4. 4th step: Implementing the study plan After deciding upon the type of study and defining the protocol, a Study Operations Manual providing detailed instructions for every single study procedure has to be compiled. This should
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
include the step-by-step process for enrolling and following patients, entering and managing data, and monitoring the process. Copies of all study materials, including study protocol, consent forms (in all the languages used), questionnaires, etc. should also be included in the manual. Procedures for maintaining confidentiality and quality assurance and control should also be covered. Here one should always consider and incorporate all relevant and applicable facts from the so-called Good Clinical Practice (GCP) procedures. This is an international ethical and scientific quality standard for designing, conducting, recording, and reporting trials that involve the participation of human subjects (www.emea.eu.int/pdfs/human/ich/013595en.pdf). When implementing the study plan it is important to filter the IMPORTANT aspects of the GCP-guidelines and to present them in a USER FRIENDLY format. 2.5. 5th step: Appraisal of results, report writing and submission for publication a) Appraisal of results Avoid data dredging and best test seeking. With data dredging, multiple possible associations in the data are tested in the hope of finding something significant. This gives rise to spurious associations. With best test seeking, the investigator seeks out good tests instead of good associations, which tends to overstate the statistical significance of an association. The basic data have to be properly described (age, sex, deformity/disease, type of practice/investigation set-up). This is required to generalise the study findings to the surgical practice of the reader (external validity). The measures of effect and the statistical significance have to be properly presented. The benefit offered by the object of the study, THE NEW OPTION, e.g. the risk reduction should be clearly presented. “Absolute” risk reduction is the risk in the control group minus the risk in the treatment group. “Relative” risk reduction is the absolute risk reduction divided by the risk in the control group, expressed as percentage.
101
One should focus on the main finding and not emphasize a finding in a subgroup, when this was not planned for a priori. Try to eliminate any bias and to understand any confounders. Bias can systematically deviate the results away from the truth. Confounders are variables that are responsible for an apparent, but false, association between two study variables. b) Report writing and submission for publication First of all one has to decide which audience is to be addressed. This will then lead to the journals that are most appropriate. In order to choose which of these, you might visit the website of the journals and look at their tables of content in order to find out about their mission and interest. It may also be of interest to see how long the intervals are between submission, acceptance, and publication in these journals. It may be important to learn about the current impact factor of the journals in question. The editorin-chief may be asked for this information (e-mail address on publisher’s website). Once the journal has been decided upon, one should always inquire about its guidelines of formatting and submission. So-called “instructions for authors” are to be found in printed issues of that journal. When writing for scientific publication, as a rule, one should always start with an Introduction that informs the reader about the relevant area and the problem this paper is tackling, the reason why this is an important problem, and how the investigation was performed. The Material and Methods section should describe the type of study (e.g. cohort study), all the inclusion and exclusion criteria, the period during which the data has been collected, the institutions involved, the descriptive analytical data of the patients/material collected should be described (including age, sex, recruitment, enrolment, ways of application of the various treatments, loss to follow-up, definition of outcome criteria, and how determined or measured, analysed, and compiled). This should be supplemented by the statistical tests applied e and the reliability and the precision of data measurement, collection, and analysis.
Table 3 Reporting quality checklist (based on the CONSORT checklist; after Moher et al., 2001). Paper selection & topic
CONSORT item number
Descriptor
Title & Abstract Methods Participants
1
The study should be identifiable as a RCT.
3
Intervention
4
Objectives Outcome
5 6
The planned study population should be defined. Inclusion and exclusion criteria should be defined. Each intervention should be fully described, including name of products, name of producer, city, and country. An aim or (null) hypothesis should be stated. Primary and secondary outcome measures should be defined. Measurement errors should be reported. How was the sample size determined? The method used to generate the random allocation sequence should be stated. The method used to implement the random allocation sequence should be stated. How was allocation sequence concealed from executor of assignment? How were patients blinded to group assignment? Were the investigator and/or assessor blinded to the assignment? The statistical methods used to compare groups for primary outcome should be stated.
Sample size Sequence generation Allocation concealment Implementation Blinded participants Blinded investigator/assessor Statistical methods Results Participant flow Baseline data Number analysed Adverse effects Discussion Interpretation
7 8 9 10 11a 11b 12
13 15 16 19 20
A flow diagram or description of non-compliance/drop out has to be provided/is very helpful Demographic data have to be included Was the analysis based on “intention-to-treat”? All complications should be described Limitations such as bias, possible confounding factors and imprecision should be discussed
102
M.Y. Mommaerts et al. / Journal of Cranio-Maxillo-Facial Surgery 40 (2012) 97e102
Quality of reporting is very important when dealing with RCTs. Sample size calculation, randomisation methodology, allocation concealment and blind investigation/assessment are often not adequately reported or are omitted (Schulz et al., 1995; Karri, 2006). In the Results section all the results obtained including the descriptive and the analytical statistical results should be listed. Here tables will often help. Raw data should be kept available for an interested reader, and this can be in different formats (table, attachment, or available upon request). And in the Discussion the results are compared with those reported by others in the scientific literature. The relevancies of statistical and of clinical significances should be discussed. Credibility and logical connections should also be checked: “Does it make sense?” One should avoid only citing supportive data. Finally a paragraph on the implications of the results obtained should follow, together with indications for generalisation and for future research in the area. Honour requires that the weaknesses and limitations of the study are pointed out before ending with a short and concise conclusion. The quality of reporting can be improved by following the CONSORT statement (Moher et al., 2001; Karri, 2006; Table 3). The Conclusion is the last section with a short summary of the most important findings and their inference for diagnosing or treating the disorder in question. Here, any speculation should be avoided. Linguistic corrections: Authors whose mother tongue is not English often underestimate the importance of correct English in the manuscript, and think that the technical editor will take care of it. In Europe, most of the reviewers are also of non-English origin and understanding the text is the first prerequisite for judging the quality of the content. The quality of the language and of the formatting unfortunately do play a role in deciding which attitude a reviewer unconsciously, or consciously has towards a manuscript. Companies specialised in editing, writing and proofreading can be of help (sometimes even great help), although often at a significant cost. When the paper has been returned to the authors from the Editor-in-Chief, there will almost never be a prompt acceptance of the paper in the format it was submitted. There will most often be some kind of criticism. This should be interpreted as a helpful way of improving the paper. Apart from linguistic shortcomings, comments involve mainly the scientific basis of the interpretation of data in the discussion. If the criticism is not justified or even unfair, it is alright to answer with a letter of polite dissent disputing the editorial board members’ opinions. If it was rejected the manuscript should not just be dropped. Rather it should be corrected, improved and then either resubmitted or sent to another journal. Perseverance and patience are required. Of the 97 manuscripts reviewed for the journal “Plastic and Reconstructive Surgery” by one reviewer between 1992 and 2003, not one was regarded to be “acceptable as is”, of 44 the reviewer felt that “major revision was needed”, only 44 were published in the targeted journal, whilst 22 were published in a PubMed-indexed journal other than the one targeted (Loonen et al., 2005). 3. Conclusion This paper is a guide for young residents who want to plan, perform and publish a study on their new clinical research in a peer
reviewed journal. The main emphasis is laid on the thoughts and techniques necessary for designing, pursuing and writing of a randomised control trial. But this article may also help in producing other kinds of manuscripts, e.g. for preparing a clinical observational study. Conflict of interests notification There is no other interest or conflict of interests to be disclosed for anyone of the three authors. List of sponsors No grants were received for this scientific paper and no sponsor was supporting it either. References Barton S: Which clinical studies provide the best evidence? Br Med J 321: 255e256, 2000 Bellomo R, Bagshaw SM: Evidence-based medicine: classifying the evidence from clinical trials - the need to consider other dimensions. Crit Care 10: 1-8, 2006 (available online: http://ccforum.com/content/10/5/232) Brooke Steward S, Oeppen RS, Cascarini L, Brennan PA: Educational article: what gets accepted for presentation? e A study of submitted abstracts for the 2009 BAOMS conference. Br J Oral Maxillofac Surg 48: 297e300, 2010 Doll R, Hill AB: Smoking and carcinoma of the lung. Br Med J 2(4682): 739e748, 1950 D’Souza G, Kreimer AR, Viscidi R, Pawlita ML, Fakhry C, Koch WM, et al: Casecontrol study of human papilloma virus and oropharyngeal cancer. N Engl J Med 356: 1944e1956, 2007 Ellis J, Mulligan J, Rowe J, Sackett DL: Inpatient general medicine is evidence based. A-Team, Nuffield Department of Clinical Medicine. Lancet 346: 407e410, 1995 Emanuel EJ, Miller FG: The ethics of placebo-controlled trials. N Engl J Med 346: 382e383, 2002 Evans M: Writing a paper. Br J Oral Maxillofac Surg 45: 485e487, 2007 Horng S, Miller FG: Is placebo surgery unethical? N Engl J Med 347: 137e139, 2002 Horton R: Surgical research or comic opera: questions, but few answers. Lancet 347: 984e985, 1996 Howes N, Chagla L, Thorpe M, McCulloch P: Surgical practice is evidence based. Br J Surg 84: 1220e1223, 1997 Karri V: Randomised clinical trials in plastic surgery: survey of output and quality of reporting. J Plast Reconstr Aesthet Surg 59: 787e796, 2006 Loonen MPJ, Hage JJ, Kon M: Who benefits from peer review? An analysis of the outcome of 100 requests for review by plastic and reconstructive surgery. Plast Reconstr Surg 116: 1461e1472, 2005 MacLehose RR, Reeves BC, Harvey IM, Sheldon TA, Russell IT, Black AM: A systematic review of comparisons of effect sizes derived from randomised and nonrandomised studies. Health Technol Assess 4: 1e154, 2000 McKibbon A, Eady A, Marks S: PDQ-evidence-based principles and practice. Hamilton, Ontario (Canada): BC Decker, 1999 Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357: 1191e1194, 2001 Panesar SS, Thakrar R, Athanasiou T, Sheikh A: Comparison of reports of randomised controlled trials and systematic reviews in surgical journals: literature review. J R Soc Med 99: 470e472, 2006 Sackett DL, Rosenberg WMC, Gray JAM, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. Br Med J 312: 71e72, 1996 Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273: 408e412, 1995 Slim K: Limits of evidence-based surgery. World J Surg 29: 606e609, 2005 Solomon MJ, McLeod RS: Should we be performing more randomized controlled trials evaluating surgical operations? Surgery 118: 459e467, 1995 Ulmer H-V: Differenz erfragen. In Diskussion zu. In: Victor A, Elsässer A, Hommel G, Blettner M (eds) Wie bewertet man die p-Wert-Flut? Dtsch Arztebl, vol. 107; 2010, 417, 2010 Urschel JD: How to analyze an article. World J Surg 29: 557e560, 2005 Weijer C: The ethics of placebo-controlled trials. J Bone Miner Res 18: 1150e1153, 2003 Wente MN, Seiler CM, Uhl W, Büchler MW: Perspectives of evidence-based surgery. Dig Surg 20, Epub 2003.