- Email: [email protected]
The use of meta-analytical tools in risk assessment for food safety
Food Microbiology 28 (2011) 823e827
Contents lists available at ScienceDirect
Food Microbiology journal homepage: www.elsevier.com/locate/fm
The use of meta-analytical tools in risk assessment for food safety Ursula Gonzales-Barron*, Francis Butler Biosystems Engineering, UCD School of Agriculture, Food Science and Veterinary Medicine, University College Dublin, Dublin 4, Ireland
a r t i c l e i n f o
a b s t r a c t
Article history: Available online 21 April 2010
This communication deals with the use of meta-analysis as a valuable tool for the synthesis of food safety research, and in quantitative risk assessment modelling. A common methodology for the conduction of meta-analysis (i.e., systematic review and data extraction, parameterisation of effect size, estimation of overall effect size, assessment of heterogeneity, and presentation of results) is explained by reviewing two meta-analyses derived from separate sets of primary studies of Salmonella in pork. Integrating different primary studies, the first meta-analysis elucidated for the first time a relationship between the proportion of Salmonella-carrier slaughter pigs entering the slaughter lines and the resulting proportion of contaminated carcasses at the point of evisceration; finding that the individual studies on their own could not reveal. On the other hand, the second application showed that meta-analysis can be used to estimate the overall effect of a critical process stage (chilling) on the incidence of the pathogen under study. The derivation of a relationship between variables and a probabilistic distribution is illustrations of the valuable quantitative information synthesised by the meta-analytical tools, which can be incorporated in risk assessment modelling. Strengths and weaknesses of meta-analysis within the context of food safety are also discussed. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Meta-analysis Risk assessment Salmonella Slaughter Pig Pork Chilling
1. Introduction Meta-analysis refers to the statistical analysis of a large collection of results from individual studies, such as experimental studies, opinion surveys and causal models, for the purpose of integrating the findings (Glass, 1976). Although interest in synthesizing findings dates back to the work of Yates and Cochran (1938) who combined results of different agricultural studies, it appears however that the sole introduction of a term for this collection of studies (‘meta-analysis’ as coined by Glass, 1976) led to an upsurge in the development and application, principally in medicine and social sciences. The primary aim of meta-analysis is to produce a more precise estimate of the effect of a particular intervention or treatment, with an increased statistical power. Since different primary studies are performed using different populations, different designs and a whole-range of other specific factors, it has been suggested that combining them would produce an estimate that has broader generalisability than is possible using only a single study (Sutton et al., 2001). Additionally, meta-analysis can be used to get an insight of the sources of heterogeneity or differences among the results of the primary research. In this sense, meta-analysis not only investigates the reported results of the
* Corresponding author. Tel.: þ353 1 7167450; fax: þ353 1 7167415. E-mail address: [email protected] (U. Gonzales-Barron). 0740-0020/$ e see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.fm.2010.04.007
studies but all aspects of research designs that produced them, such as theoretical constructs, operational definitions of variables, population samples, data collection procedures, statistical analysis, and especially the handling of possible confounding variables that would provide an alternative explanation for the reported results (Noble, 2006). As stated by Sargeant et al. (2006), the prevention of food-borne illnesses is complex because of the multiple stages in the production and preparation of food. On the other hand, the amount of data produced by food safety research have been growing increasingly in the last ten years, and the advances in information technology are likely to further contribute to this growth. Therefore, there is a need for conducting meta-analysis in the field of food safety, to identify, evaluate and synthesize results, so that policy-makers can access evidence-based and concise information on the effectiveness of interventions to control and prevent food-borne illnesses in humans (Sargeant et al., 2005). Although, in principle, meta-analysis may be conducted to address a broad range of food safety research questions such as effect of interventions pre-harvest (for instance, interventions to reduce faecal shedding of Escherichia coli O157 in beef cattle), effect of interventions post-harvest (for instance, effect of carcass rinsing on Salmonella), disease incidence, prevalence of pathogens, consumer practices, etc., applications in food safety research are still in its infancy. The findings of such independent meta-analyses can offer valuable information on the best interventions and can provide data as input into risk assessment models.
824
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827
To date, only six published studies using meta-analysis as a tool to combine food safety data have been identified (Patil et al., 2004; Vialette et al., 2005; Sánchez et al., 2007; Bollaerts et al., 2008; Gonzales-Barron et al., 2008; Gonzales-Barron et al., 2009). This article aims to (i) present the general objectives and methodology of meta-analysis and its relevance for the synthesis of food safety research by reviewing two applications, and (ii) to highlight its use in risk assessment modelling discussing its strengths and weaknesses. 2. Methodology of meta-analysis While building a risk assessment model for predicting Salmonella prevalence during pork production in Ireland, GonzalesBarron et al. (2008, 2009) identified, (i) ten primary studies that reported incidence of Salmonella recovery from post-chill pig carcasses (sT/nT ¼ pT) in comparison to their recovery after splitting and rinsing (sR/nR ¼ pR). The outcome data were available on nT pig carcasses in the post-chill or treated group and nR pig carcasses in the pre-chill control group. The numbers of successes (Salmonella-positive carcasses) are represented by sT and sR for the chilled and prechill (control) group, respectively. (ii) ten primary studies that individually reported proportions of Salmonella-positive cecal contents (sC/nC ¼ pC) from batches of pigs sampled entering the slaughter lines and Salmonellapositive eviscerated pig carcasses (sE/nE ¼ pE) sampled from same production batches. In order to obtain more-informed overall effect size estimates, both data sets were separately meta-analysed using a common methodology, whose stages are explained below. They will be referred to as ‘slaughter meta-analysis’ and ‘chilling meta-analysis’, respectively.
2.1. Systematic review The first step in performing a meta-analysis is to carry out a systematic review, whose primary aim is to produce a summary of all available studies, assessed for relevance, quality and reported findings. The purpose of screening for relevance is to identify those articles that specifically may help to address the meta-analysis question(s). At this step, it is essential that decisions about inclusion or exclusion of studies are made according to predetermined criteria in order to prevent any bias indecision-making. The relevance screening tool generally consists of a short series of questions which are designed to quickly determine whether or not the article belongs in the meta-analysis. The questions should be specific and defined based on the population, the intervention, the outcomes and the study designs of interest. Since the validity of meta-analysis depends on the quality of the systematic review on which it is based, the researcher must aim to identify, appraise, select and synthesize all high quality information and data relevant to the question to date. A well conducted meta-analysis will produce poor results if the primary studies were poorly carried out. Thus, the selected primary studies should undergo critical appraisal to assess their validity. It is necessary to determine a minimum quality threshold for inclusion of research findings in the meta-analysis. The purpose of the quality assessment step is to exclude studies whose quality is too low to provide meaningful data to address the review question. Quality assessment instruments are usually grouped under generic and specific items (Sargeant et al., 2005). Generic quality items include objectives and study population, intervention, outcome assessment, withdrawals and loss to follow-up and data analysis
and control for confounders. Individual aspects of study methodology are incorporated within the generic components for all study designs, for example information on precision of the study, external validity, randomization, comparison groups, blinding of intervention status, loss to follow-up, assessment of relevant outcomes, and statistical analysis. It is important to consider these aspects because they have a potential relation to bias in estimation of effects. A systematic review begins with the formulation of a focused study question for which three important facets are to be considered: population, intervention or treatment and outcome. For instance, in the chilling meta-analysis, the problem statement was the estimation of the overall effect of chilling on the Salmonella prevalence of pig carcasses during pork production. The population was specified as eviscerated pig carcasses post-meat inspection in slaughterhouses. The intervention was represented by the chilling stage during pork processing, which includes cooling and posterior cold storage (18e24 h) at w5 C. The measured outcome was the presence of Salmonella spp. on the pig carcass surface. The two principal questions addressed in both meta-analyses were: (i) is there any support in the sampled population of studies for the causal inference that the intervention (slaughter/chilling) made a statistically-significant difference in the outcome (presence of Salmonella on a pig carcass)? And if so (ii) how large an effect or difference did the intervention make? 2.2. Data extraction The data extraction from the primary studies should provide the information necessary for summarizing and synthesizing the results, and includes both numeric and non-numeric data. The data to be extracted from each primary study should include information on the study characteristics related to the population, intervention, outcome and study design, and the results, including features such as length of follow-up, effect measures and variability in the effect measures. Details of the intervention and how it was administered to the population should be included because differences between intervention protocols may be an important source of heterogeneity among studies. It is also important to record what was given to the control group(s). Different types of control groups can be a source of differences among studies. Control groups may include non-treated controls, placebo-treated controls, or controls given an alternative treatment protocol. Information on the details of the laboratory technique(s) used to measure the outcome of interest (such as culture, serological, biochemical or molecular tests), the use of positive and negative controls, and the diagnostic criteria is important for descriptive purposes and as a potential source of heterogeneity between studies (Sargeant et al., 2005). 2.3. Parameterisation of the effect size Effect size refers to the degree to which the phenomenon is present in the population (for example, decrease in the recovery of Salmonella on pig carcasses due to chilling). For the primary studies to be compatible to analyze, meta-analysis converts the effect size into a ‘parameter’ that allows direct comparison and summation of the primary studies. Depending on the type of data, there are many types of effect size parameters. Broadly speaking, quantitative outcomes can be classified as (i) binary or dichotomous, e.g., indicating the presence or absence of the event of interest in each subject, (ii) continuous, and (iii) ordinal, where the outcome is measured on an ordered categorical scale (Table 1). It is important to select, however, the appropriate parameter to describe and summarize the data because different effect size parameters lead to
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827 Table 1 Overview of commonly used numerical and graphical meta-analytical techniques. Numerical meta-analytical techniques
Graphical meta-analytical tools
Outcomes measure e parameterisation: Binary data: Odds ratio, risk ratio, risk difference Continuous data: Mean difference, standardised mean (Cohen's d), bias-corrected standardised mean difference (Hedges' g) Correlation data: Fisher's Z Individual studies analysis: Weights p-values Fixed-effects meta-analysis: Inverse variance weighted method Maximum likelihood techniques (REML) Mantel-Haenszel, Peto Heterogeneity: Cochran's Q, Higgins' H and I2 Hedges and Olkin test of homogeneity for correlation coefficients Random-effects meta-analysis: DerSimonian-Laird method Weighted and normal-normal method Meta-regression: Mixed-effect models Multi-level models Publication bias: Trim-and-fill test, fail-safe N Egger's regression test Begg's rank correlation test Macaskill's regression test Bayesian methods: Normally distributed data, binary data Assessing study quality and publication bias Hierarchical models
Forest plot Funnel plot Bubble plot Plot of normalized z-scores Radial plot (Galbraith plot) L'Abbe plot Baujat plot Egger's regression plot Macaskill's regression plot Trim-and-fill plot
825
Both meta-analyses assigned precision to the individual effect size using the common method of weighting individual estimates by means of their inverse within-study variances (u ¼ 1/Var(RR)) (Table 1). Thus, a primary study with low variance (higher precision) will have more influence in the overall estimate as compared with a primary study with high variance (lower precision). The null hypothesis of absence of effect on the ‘treated’ group was tested using the U statistic (U ¼ S(RRi$ui)2/Sui) with the chi-squared distribution with one degree of freedom. At the end of this step, for each of the primary studies, estimates of effect size, standard error, weight and confidence intervals will have been calculated as well as the overall effect size, its variance and confidence interval (Gonzales-Barron et al., 2008). 2.5. Assessment of heterogeneity among primary studies
different meta-analysis results. When dichotomous data are used, the effect size can be expressed as odd ratios, risk ratios or risk difference. With continuous data, the effect size can be summarised as means, mean differences or standardised means. Studies also may use survival or time to event data where the outcome is the time to the occurrence of an event and the results are reported using hazard ratios. In both meta-analyses, the effect size was parameterised as the ‘relative risk’ (RR), defined as probability of outcome of interest in the treatment group relative to the probability of outcome of interest in the control group (for the slaughter meta-analysis, RR ¼ pE/pC, and for the chilling meta-analysis, RR ¼ pT/pR).
In food research, the fixed-effect assumption may be unrealistic given the variability of the biological systems, and also the differences in study protocols. In addition, even if the same protocols are used in all primary studies, variability in study quality may give rise to heterogeneity. If effect size estimates vary between studies to a greater extent than expected on the basis of chance alone (fixedeffect), the studies are considered heterogeneous and it is necessary to account for the extra-variation in the meta-analysis model. A simple method to assess heterogeneity is by using the Q statistic, P 2 Q ¼ RRi ui U, to test the null hypothesis of absence of heterogeneity. The way this is usually done is through the use of a randomeffects model (Table 1). Basically, it relaxes the assumption that each primary study is estimating exactly the same underlying effect size, and instead includes another random component by assuming that the true effect size in each study is itself a realization of a random variable. In a random-effects meta-analysis, each primary study is also weighted (u*) by the inverse of its variance, with the difference that the variance now includes the original within-study variance Var(RR) plus an estimate of the between-studies variance s2 (u* ¼ 1/ [Var(RR) þ s2]). The most common method for estimating s2 is the method of moments or DerSimonian and Laird method (Table 1): P P P s2 ¼ ½Q ðr 1Þ=½ ui u2i = 6i (r ¼ number of primary studies). This method is easy to compute although does not make any assumption about the distribution of the random effects. Alternatives exist and some statisticians favour the restricted maximum likelihood method (Table 1). After s2 has been estimated, weights, standard errors and the overall effect size must be recalculated. Heterogeneity in each of the meta-analyses was assessed using the Q statistic distributed as a chi-squared with r 1 degrees of freedom.
2.4. Estimation of the overall effect size
2.6. Presentation of meta-analysis results
The next step is to combine the primary studies to compute the overall effect size estimate using first a fixed-effect approach. The fixed-effect meta-analysis makes the strong assumption that each study is estimating the same underlying treatment effect, and that all differences in observed effects are due to sampling error. In other words, if each primary study had an infinite sample size, the sampling error would be zero and the observed effect for each study would be the same as the true effect. In order to obtain the most precise estimate of the population effect size, primary studies may be weighted to reflect sample size, quality of research design or other factors influencing their reliability. A dominant factor in precision is the sample size, with larger samples yielding more precise estimates than smaller samples. Another factor affecting precision is the study design, with matched groups yielding more precise estimates (as compared with independent groups) and clustered groups yielding less precise estimates.
There are a series of graphical displays to present meta-analysis results (Table 1) although the most common way to summarize the primary studies and overall effect size is by using a ‘forest plot’ which displays point estimates and confidence intervals of each primary study and the overall fixed-effect and/or random-effects estimate. A mechanism for displaying the relationship between study size and effect size is the funnel plot. The funnel plot is plotted with effect size on the x-axis and the inverse of the variance or standard error on the y-axis. In the absence of publication bias, the studies will be distributed symmetrically about the mean effect size. A bubble plot is used extensively in meta-regression to represent the relationship between a covariate or moderator and a dependent variable, with the bubble size proportional to the precision or weight given to the primary study. The results from the slaughter meta-analysis and the chilling meta-analysis were displayed as a bubble plot and a funnel plot, respectively.
826
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827
3. Results and discussion The first meta-analysis demonstrates how the synthesis of primary studies can help reveal a relationship between two variables. In the slaughter meta-analysis, some of the primary studies reported a statistical association (found by chi-square tests) between the proportion of Salmonella-positive cecal contents (pC ¼ sC/nC) from pigs entering the slaughter lines and the proportion of positive carcass swabs (pE ¼ sE/nE) sampled from the same production batches; although no relationship at batch level (or groups of slaughter) was shown in any of the primary studies. Since these binary data were extracted from studies that used different protocols for Salmonella culture, it was necessary to allow for the differences by correcting the underestimated pC and pE values with test sensitivities for cecal culture and carcass swabs (producing then pC0 and pE0 ). While the meta-analysis elucidated for the first time a relationship between pC0 and pE0 (U ¼ 372; p < 0.001; Fig. 1), these data pairs could not be combined into a simple least square regression since they had been extracted from a series of primary studies presenting various degrees of precision. On the other hand, slaughter procedures may not be necessarily uniform across studies. Thus, the utilization of a meta-analytical tool for assigning weights to the data pairs (pC0 , pE0 ) was considered suitable in this case. Weights were calculated for each primary study as the inverse of the variance of the RR, defined as the probability of producing Salmonella-positive eviscerated carcasses relative to the probability of Salmonella-carrier slaughter pigs entering the slaughter lines. As the Q test gave evidence of presence of heterogeneity (Q ¼ 29.3; p < 0.001), all weights were recalculated for a random-effects solution (Gonzales-Barron et al., 2009). The relative size of the bubbles shown in Fig. 1 is proportional to the weight assigned to each primary study. In summary, this metaanalysis has shown how the body of information contained in all studies revealed a clearer picture of the state of knowledge. In Gonzales-Barron et al. (2009), this analysis was combined with a non-parametric bootstrap technique in order to build a stochastic regression between the Salmonella carriage rate in pigs entering the abattoirs and the resulting incidence on carcasses at the point of evisceration. The second case demonstrates the application of meta-analysis for the estimation of the overall effect of a critical process stage on the incidence of a pathogen under study. Specifically, the overall effect size of chilling on the recovery of Salmonella from pork carcasses was estimated. The chilling meta-analysis was conducted
on the RR measure (defined as the probability of encountering Salmonella-positive pig carcasses after chilling relative to the probability of encountering Salmonella-positive carcasses before chilling). The chilling meta-analysis confirmed on the grounds of increased statistical power the decreasing effect that chilling has on the recovery of Salmonella (U ¼ 27.3; p < 0.001). As there was no evidence of heterogeneity (Q ¼ 2.9; p ¼ 0.96), the fixed-effects meta-analysis was considered a suitable solution and it delivered a normal distribution of the overall effect of chilling (with a mean of w2.4 times reduction in Salmonella recovery on pig carcasses, Fig. 2). The parameterisation of relative risk was believed to be a more appropriate statistical measure for this meta-analysis because, being essentially a ratio of probability after chilling to probability before chilling, the different sensitivity values due to detection test, protocols and extent of swabs will be rather cancelled off in every primary study estimate, and this will therefore contribute to a better agreement among primary studies (since less sources of heterogeneity are affecting the effect size measure). This fact made the overall relative risk estimate of meta-analysis more generalisable, in the sense that the overall reduction factor of w2.4 in Salmonella prevalence can be extended to the broad range of tests that were used in the studies to determine Salmonella presence in pig carcasses. Because meta-analysis relies on actual data, the effect size outcome distribution can be used instead of, or in addition to, expert judgment in quantitative risk assessment models. Hence, it is expected that the normal distribution of the effect size of chilling on Salmonella prevalence will provide a more precise and realistic input distribution of the chilling stage for risk assessment models. Meta-analysis, if carefully constructed and implemented, can assist food safety researchers to determine the extent to which accumulated evidence tends to provisionally confirm or conclusively refute a specified theory of the intervention under investigation. The claimed strengths of meta-analysis are all contingent on the important condition that the measures that represent and share the same theoretical concepts within a meta-analysis be of at least satisfactory validity (related to the principle of ‘rubbish in e rubbish out’). The main limitations of meta-analysis are related to quality of primary research and publication bias (Noble, 2006). Meta-analysis is incapable of correcting the limitations imposed by underpowered primary studies that introduce small sample bias with overstatement of the estimated effect size as well as insensitivity to the influence of relevant moderators on outcomes. Underpowered
4.5
Precision (1/standard error )
Proportion of resulting Salmonellapositive eviscerated carcasses
0.40 0.35 0.30 0.25 0.20 0.15 0.10 2
R = 0.770
0.05 0.00
4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
0
0.2
0.4
0.6
0.8
Proportion of slaughter pigs carrrying Salmonella in caeca Fig. 1. Bubble plot of the cecal Salmonella carriage rate in slaughter pigs and the resulting Salmonella-positive eviscerated carcasses. Bubble area is proportional to the weight assigned to each primary study.
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
Log relative risk (log p T /pR) Fig. 2. Funnel plot of the relative risk of the Salmonella recovery on pig carcasses after chilling (pT) in relation to before chilling (pR). Vertical line at log pT/pR ¼ 0.88 (pR/pT ¼ w2.4) represents the overall effect size, and marker area is proportional to the weight assigned to each primary study.
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827
primary research, that is, research that employs samples of insufficient size and insufficient variability, simultaneously constrains detections of small statistically-significant differences between intervened and control subjects, and increases the likelihood of accepting the null hypothesis as true when it is false (type II error). On the other hand, publication bias exists because research with statistically-significant results is potentially more likely to be published than work with non-significant results. The presence of publication bias in a meta-analysis can be assessed informally by inspection of a funnel plot, which plots the effect size of each study against some measure of its precision (1/standard error). The resulting plot should be shaped like a funnel if no publication bias is present. This shape is expected because trials of decreasing size have increasingly large variation in their effect size estimates due to random variation becoming increasingly influential. However, if the chance of publication is greater for larger trials or trials with statistically-significant results, some small non-significant studies may not appear in the literature, leading to the omission of trials in one corner of the plot. Fig. 2 shows a funnel plot of the primary studies used to conduct the chilling meta-analysis. Visual inspection would suggest that, despite the few studies utilized for this meta-analysis, there is little evidence of publication bias in this dataset. 4. Conclusions While meta-analysis is not without limitations (underpowered primary research and publication bias), its systematic approach comprising a broad range of techniques merits consideration among food safety researchers to integrate the current body of knowledge and data on targeted issues along the complex continuum of agro-food production, and to furnish increased credibility to findings in the field. As reviewed in the two metaanalysis applications, while primary studies might be weak or reach contradictory conclusions, the body of information contained in all studies might reveal a clearer picture of the state of knowledge, and can offer valuable quantitative information of effect size in the form of distributions that can be inserted into risk assessment models. Meta-analysis can highlight areas where there is insufficient
827
evidence of the efficacy of interventions, where there is absence of high quality studies, or where there are common methodological flaws in the available research, and can therefore provide direction for future research. Acknowledgments The authors wish to acknowledge Safe Food, The Food Safety Promotion Board and the Food Institutional Research Measure (FIRM) administered by the Irish Department of Agriculture, Fisheries and Food. References Bollaerts, K., Aerts, M., Faes, C., Grijspeerdt, K., Dewulf, J., Mintiens, K., 2008. Human salmonellosis: estimation of dose-illness from outbreak data. Risk Anal. 28 (2), 427e440. Glass, G.V., 1976. Primary, secondary and meta-analysis of research. Educ. Res. 5 (1, 3), 3e8. Gonzales-Barron, U., Bergin, D., Butler, F., 2008. A meta-analysis study of the effect of chilling on Salmonella prevalence on pork carcasses. J. Food Prot. 71 (7), 1330e1337. Gonzales-Barron, U., Soumpasis, I., Butler, F., Duggan, S., Prendergast, D., Duffy, G., 2009. Estimation of prevalence of Salmonella spp. on pig carcasses and pork joints using a quantitative risk assessment model aided by meta-analysis. J. Food Prot. 72 (2), 274e285. Noble, J.H., 2006. Meta-analysis: methods, strengths, weaknesses and political uses. J. Lab. Clin. Med. 147 (1), 7e20. Patil, S.R., Morales, R., Cates, S., Anderson, D., Kendall, D., 2004. An application of meta-analysis in food safety consumer research to evaluate consumer behaviours and practices. J. Food Prot. 67 (11), 2587e2595. Sánchez, J., Dohoo, I.R., Christensen, J., Rajic, A., 2007. Factor influencing the prevalence of Salmonella spp. in swine farms: a meta-analysis approach. Prev. Vet. Med. 81, 148e177. Sargeant, J.M., Amezcua, M., Rajic, A., Waddell, L., 2005. A Guide to Conducting Systematic Reviews in Agri-food Public Health. Available at:. Food Safety Research and Response Network, Canada http://www.fsrrn.net/UserFiles/File/ conductingsysreviewsenglish[1].pdf. Sargeant, J.M., Rajic, A., Read, S., Ohlsson, A., 2006. The process of systematic review and its application in agri-food public health. Prev. Vet. Med. 76, 141e151. Sutton, A.J., Abram, K.R., Jones, D.R., 2001. An illustrated guide to the methods of meta-analysis. J. Eval. Clin. Pract. 7 (2), 135e148. Vialette, M., Pinon, N., Leporq, B., Dervin, C., Membré, J.M., 2005. Meta-analysis of food safety information based on a combination of a relational database and a predictive modelling tool. Risk Anal. 25 (1), 75e83. Yates, F., Cochran, W.G., 1938. The analysis of groups of experiments. J. Agric. Sci. 28 (1), 556e580.
Contents lists available at ScienceDirect
Food Microbiology journal homepage: www.elsevier.com/locate/fm
The use of meta-analytical tools in risk assessment for food safety Ursula Gonzales-Barron*, Francis Butler Biosystems Engineering, UCD School of Agriculture, Food Science and Veterinary Medicine, University College Dublin, Dublin 4, Ireland
a r t i c l e i n f o
a b s t r a c t
Article history: Available online 21 April 2010
This communication deals with the use of meta-analysis as a valuable tool for the synthesis of food safety research, and in quantitative risk assessment modelling. A common methodology for the conduction of meta-analysis (i.e., systematic review and data extraction, parameterisation of effect size, estimation of overall effect size, assessment of heterogeneity, and presentation of results) is explained by reviewing two meta-analyses derived from separate sets of primary studies of Salmonella in pork. Integrating different primary studies, the first meta-analysis elucidated for the first time a relationship between the proportion of Salmonella-carrier slaughter pigs entering the slaughter lines and the resulting proportion of contaminated carcasses at the point of evisceration; finding that the individual studies on their own could not reveal. On the other hand, the second application showed that meta-analysis can be used to estimate the overall effect of a critical process stage (chilling) on the incidence of the pathogen under study. The derivation of a relationship between variables and a probabilistic distribution is illustrations of the valuable quantitative information synthesised by the meta-analytical tools, which can be incorporated in risk assessment modelling. Strengths and weaknesses of meta-analysis within the context of food safety are also discussed. Ó 2010 Elsevier Ltd. All rights reserved.
Keywords: Meta-analysis Risk assessment Salmonella Slaughter Pig Pork Chilling
1. Introduction Meta-analysis refers to the statistical analysis of a large collection of results from individual studies, such as experimental studies, opinion surveys and causal models, for the purpose of integrating the findings (Glass, 1976). Although interest in synthesizing findings dates back to the work of Yates and Cochran (1938) who combined results of different agricultural studies, it appears however that the sole introduction of a term for this collection of studies (‘meta-analysis’ as coined by Glass, 1976) led to an upsurge in the development and application, principally in medicine and social sciences. The primary aim of meta-analysis is to produce a more precise estimate of the effect of a particular intervention or treatment, with an increased statistical power. Since different primary studies are performed using different populations, different designs and a whole-range of other specific factors, it has been suggested that combining them would produce an estimate that has broader generalisability than is possible using only a single study (Sutton et al., 2001). Additionally, meta-analysis can be used to get an insight of the sources of heterogeneity or differences among the results of the primary research. In this sense, meta-analysis not only investigates the reported results of the
* Corresponding author. Tel.: þ353 1 7167450; fax: þ353 1 7167415. E-mail address: [email protected] (U. Gonzales-Barron). 0740-0020/$ e see front matter Ó 2010 Elsevier Ltd. All rights reserved. doi:10.1016/j.fm.2010.04.007
studies but all aspects of research designs that produced them, such as theoretical constructs, operational definitions of variables, population samples, data collection procedures, statistical analysis, and especially the handling of possible confounding variables that would provide an alternative explanation for the reported results (Noble, 2006). As stated by Sargeant et al. (2006), the prevention of food-borne illnesses is complex because of the multiple stages in the production and preparation of food. On the other hand, the amount of data produced by food safety research have been growing increasingly in the last ten years, and the advances in information technology are likely to further contribute to this growth. Therefore, there is a need for conducting meta-analysis in the field of food safety, to identify, evaluate and synthesize results, so that policy-makers can access evidence-based and concise information on the effectiveness of interventions to control and prevent food-borne illnesses in humans (Sargeant et al., 2005). Although, in principle, meta-analysis may be conducted to address a broad range of food safety research questions such as effect of interventions pre-harvest (for instance, interventions to reduce faecal shedding of Escherichia coli O157 in beef cattle), effect of interventions post-harvest (for instance, effect of carcass rinsing on Salmonella), disease incidence, prevalence of pathogens, consumer practices, etc., applications in food safety research are still in its infancy. The findings of such independent meta-analyses can offer valuable information on the best interventions and can provide data as input into risk assessment models.
824
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827
To date, only six published studies using meta-analysis as a tool to combine food safety data have been identified (Patil et al., 2004; Vialette et al., 2005; Sánchez et al., 2007; Bollaerts et al., 2008; Gonzales-Barron et al., 2008; Gonzales-Barron et al., 2009). This article aims to (i) present the general objectives and methodology of meta-analysis and its relevance for the synthesis of food safety research by reviewing two applications, and (ii) to highlight its use in risk assessment modelling discussing its strengths and weaknesses. 2. Methodology of meta-analysis While building a risk assessment model for predicting Salmonella prevalence during pork production in Ireland, GonzalesBarron et al. (2008, 2009) identified, (i) ten primary studies that reported incidence of Salmonella recovery from post-chill pig carcasses (sT/nT ¼ pT) in comparison to their recovery after splitting and rinsing (sR/nR ¼ pR). The outcome data were available on nT pig carcasses in the post-chill or treated group and nR pig carcasses in the pre-chill control group. The numbers of successes (Salmonella-positive carcasses) are represented by sT and sR for the chilled and prechill (control) group, respectively. (ii) ten primary studies that individually reported proportions of Salmonella-positive cecal contents (sC/nC ¼ pC) from batches of pigs sampled entering the slaughter lines and Salmonellapositive eviscerated pig carcasses (sE/nE ¼ pE) sampled from same production batches. In order to obtain more-informed overall effect size estimates, both data sets were separately meta-analysed using a common methodology, whose stages are explained below. They will be referred to as ‘slaughter meta-analysis’ and ‘chilling meta-analysis’, respectively.
2.1. Systematic review The first step in performing a meta-analysis is to carry out a systematic review, whose primary aim is to produce a summary of all available studies, assessed for relevance, quality and reported findings. The purpose of screening for relevance is to identify those articles that specifically may help to address the meta-analysis question(s). At this step, it is essential that decisions about inclusion or exclusion of studies are made according to predetermined criteria in order to prevent any bias indecision-making. The relevance screening tool generally consists of a short series of questions which are designed to quickly determine whether or not the article belongs in the meta-analysis. The questions should be specific and defined based on the population, the intervention, the outcomes and the study designs of interest. Since the validity of meta-analysis depends on the quality of the systematic review on which it is based, the researcher must aim to identify, appraise, select and synthesize all high quality information and data relevant to the question to date. A well conducted meta-analysis will produce poor results if the primary studies were poorly carried out. Thus, the selected primary studies should undergo critical appraisal to assess their validity. It is necessary to determine a minimum quality threshold for inclusion of research findings in the meta-analysis. The purpose of the quality assessment step is to exclude studies whose quality is too low to provide meaningful data to address the review question. Quality assessment instruments are usually grouped under generic and specific items (Sargeant et al., 2005). Generic quality items include objectives and study population, intervention, outcome assessment, withdrawals and loss to follow-up and data analysis
and control for confounders. Individual aspects of study methodology are incorporated within the generic components for all study designs, for example information on precision of the study, external validity, randomization, comparison groups, blinding of intervention status, loss to follow-up, assessment of relevant outcomes, and statistical analysis. It is important to consider these aspects because they have a potential relation to bias in estimation of effects. A systematic review begins with the formulation of a focused study question for which three important facets are to be considered: population, intervention or treatment and outcome. For instance, in the chilling meta-analysis, the problem statement was the estimation of the overall effect of chilling on the Salmonella prevalence of pig carcasses during pork production. The population was specified as eviscerated pig carcasses post-meat inspection in slaughterhouses. The intervention was represented by the chilling stage during pork processing, which includes cooling and posterior cold storage (18e24 h) at w5 C. The measured outcome was the presence of Salmonella spp. on the pig carcass surface. The two principal questions addressed in both meta-analyses were: (i) is there any support in the sampled population of studies for the causal inference that the intervention (slaughter/chilling) made a statistically-significant difference in the outcome (presence of Salmonella on a pig carcass)? And if so (ii) how large an effect or difference did the intervention make? 2.2. Data extraction The data extraction from the primary studies should provide the information necessary for summarizing and synthesizing the results, and includes both numeric and non-numeric data. The data to be extracted from each primary study should include information on the study characteristics related to the population, intervention, outcome and study design, and the results, including features such as length of follow-up, effect measures and variability in the effect measures. Details of the intervention and how it was administered to the population should be included because differences between intervention protocols may be an important source of heterogeneity among studies. It is also important to record what was given to the control group(s). Different types of control groups can be a source of differences among studies. Control groups may include non-treated controls, placebo-treated controls, or controls given an alternative treatment protocol. Information on the details of the laboratory technique(s) used to measure the outcome of interest (such as culture, serological, biochemical or molecular tests), the use of positive and negative controls, and the diagnostic criteria is important for descriptive purposes and as a potential source of heterogeneity between studies (Sargeant et al., 2005). 2.3. Parameterisation of the effect size Effect size refers to the degree to which the phenomenon is present in the population (for example, decrease in the recovery of Salmonella on pig carcasses due to chilling). For the primary studies to be compatible to analyze, meta-analysis converts the effect size into a ‘parameter’ that allows direct comparison and summation of the primary studies. Depending on the type of data, there are many types of effect size parameters. Broadly speaking, quantitative outcomes can be classified as (i) binary or dichotomous, e.g., indicating the presence or absence of the event of interest in each subject, (ii) continuous, and (iii) ordinal, where the outcome is measured on an ordered categorical scale (Table 1). It is important to select, however, the appropriate parameter to describe and summarize the data because different effect size parameters lead to
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827 Table 1 Overview of commonly used numerical and graphical meta-analytical techniques. Numerical meta-analytical techniques
Graphical meta-analytical tools
Outcomes measure e parameterisation: Binary data: Odds ratio, risk ratio, risk difference Continuous data: Mean difference, standardised mean (Cohen's d), bias-corrected standardised mean difference (Hedges' g) Correlation data: Fisher's Z Individual studies analysis: Weights p-values Fixed-effects meta-analysis: Inverse variance weighted method Maximum likelihood techniques (REML) Mantel-Haenszel, Peto Heterogeneity: Cochran's Q, Higgins' H and I2 Hedges and Olkin test of homogeneity for correlation coefficients Random-effects meta-analysis: DerSimonian-Laird method Weighted and normal-normal method Meta-regression: Mixed-effect models Multi-level models Publication bias: Trim-and-fill test, fail-safe N Egger's regression test Begg's rank correlation test Macaskill's regression test Bayesian methods: Normally distributed data, binary data Assessing study quality and publication bias Hierarchical models
Forest plot Funnel plot Bubble plot Plot of normalized z-scores Radial plot (Galbraith plot) L'Abbe plot Baujat plot Egger's regression plot Macaskill's regression plot Trim-and-fill plot
825
Both meta-analyses assigned precision to the individual effect size using the common method of weighting individual estimates by means of their inverse within-study variances (u ¼ 1/Var(RR)) (Table 1). Thus, a primary study with low variance (higher precision) will have more influence in the overall estimate as compared with a primary study with high variance (lower precision). The null hypothesis of absence of effect on the ‘treated’ group was tested using the U statistic (U ¼ S(RRi$ui)2/Sui) with the chi-squared distribution with one degree of freedom. At the end of this step, for each of the primary studies, estimates of effect size, standard error, weight and confidence intervals will have been calculated as well as the overall effect size, its variance and confidence interval (Gonzales-Barron et al., 2008). 2.5. Assessment of heterogeneity among primary studies
different meta-analysis results. When dichotomous data are used, the effect size can be expressed as odd ratios, risk ratios or risk difference. With continuous data, the effect size can be summarised as means, mean differences or standardised means. Studies also may use survival or time to event data where the outcome is the time to the occurrence of an event and the results are reported using hazard ratios. In both meta-analyses, the effect size was parameterised as the ‘relative risk’ (RR), defined as probability of outcome of interest in the treatment group relative to the probability of outcome of interest in the control group (for the slaughter meta-analysis, RR ¼ pE/pC, and for the chilling meta-analysis, RR ¼ pT/pR).
In food research, the fixed-effect assumption may be unrealistic given the variability of the biological systems, and also the differences in study protocols. In addition, even if the same protocols are used in all primary studies, variability in study quality may give rise to heterogeneity. If effect size estimates vary between studies to a greater extent than expected on the basis of chance alone (fixedeffect), the studies are considered heterogeneous and it is necessary to account for the extra-variation in the meta-analysis model. A simple method to assess heterogeneity is by using the Q statistic, P 2 Q ¼ RRi ui U, to test the null hypothesis of absence of heterogeneity. The way this is usually done is through the use of a randomeffects model (Table 1). Basically, it relaxes the assumption that each primary study is estimating exactly the same underlying effect size, and instead includes another random component by assuming that the true effect size in each study is itself a realization of a random variable. In a random-effects meta-analysis, each primary study is also weighted (u*) by the inverse of its variance, with the difference that the variance now includes the original within-study variance Var(RR) plus an estimate of the between-studies variance s2 (u* ¼ 1/ [Var(RR) þ s2]). The most common method for estimating s2 is the method of moments or DerSimonian and Laird method (Table 1): P P P s2 ¼ ½Q ðr 1Þ=½ ui u2i = 6i (r ¼ number of primary studies). This method is easy to compute although does not make any assumption about the distribution of the random effects. Alternatives exist and some statisticians favour the restricted maximum likelihood method (Table 1). After s2 has been estimated, weights, standard errors and the overall effect size must be recalculated. Heterogeneity in each of the meta-analyses was assessed using the Q statistic distributed as a chi-squared with r 1 degrees of freedom.
2.4. Estimation of the overall effect size
2.6. Presentation of meta-analysis results
The next step is to combine the primary studies to compute the overall effect size estimate using first a fixed-effect approach. The fixed-effect meta-analysis makes the strong assumption that each study is estimating the same underlying treatment effect, and that all differences in observed effects are due to sampling error. In other words, if each primary study had an infinite sample size, the sampling error would be zero and the observed effect for each study would be the same as the true effect. In order to obtain the most precise estimate of the population effect size, primary studies may be weighted to reflect sample size, quality of research design or other factors influencing their reliability. A dominant factor in precision is the sample size, with larger samples yielding more precise estimates than smaller samples. Another factor affecting precision is the study design, with matched groups yielding more precise estimates (as compared with independent groups) and clustered groups yielding less precise estimates.
There are a series of graphical displays to present meta-analysis results (Table 1) although the most common way to summarize the primary studies and overall effect size is by using a ‘forest plot’ which displays point estimates and confidence intervals of each primary study and the overall fixed-effect and/or random-effects estimate. A mechanism for displaying the relationship between study size and effect size is the funnel plot. The funnel plot is plotted with effect size on the x-axis and the inverse of the variance or standard error on the y-axis. In the absence of publication bias, the studies will be distributed symmetrically about the mean effect size. A bubble plot is used extensively in meta-regression to represent the relationship between a covariate or moderator and a dependent variable, with the bubble size proportional to the precision or weight given to the primary study. The results from the slaughter meta-analysis and the chilling meta-analysis were displayed as a bubble plot and a funnel plot, respectively.
826
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827
3. Results and discussion The first meta-analysis demonstrates how the synthesis of primary studies can help reveal a relationship between two variables. In the slaughter meta-analysis, some of the primary studies reported a statistical association (found by chi-square tests) between the proportion of Salmonella-positive cecal contents (pC ¼ sC/nC) from pigs entering the slaughter lines and the proportion of positive carcass swabs (pE ¼ sE/nE) sampled from the same production batches; although no relationship at batch level (or groups of slaughter) was shown in any of the primary studies. Since these binary data were extracted from studies that used different protocols for Salmonella culture, it was necessary to allow for the differences by correcting the underestimated pC and pE values with test sensitivities for cecal culture and carcass swabs (producing then pC0 and pE0 ). While the meta-analysis elucidated for the first time a relationship between pC0 and pE0 (U ¼ 372; p < 0.001; Fig. 1), these data pairs could not be combined into a simple least square regression since they had been extracted from a series of primary studies presenting various degrees of precision. On the other hand, slaughter procedures may not be necessarily uniform across studies. Thus, the utilization of a meta-analytical tool for assigning weights to the data pairs (pC0 , pE0 ) was considered suitable in this case. Weights were calculated for each primary study as the inverse of the variance of the RR, defined as the probability of producing Salmonella-positive eviscerated carcasses relative to the probability of Salmonella-carrier slaughter pigs entering the slaughter lines. As the Q test gave evidence of presence of heterogeneity (Q ¼ 29.3; p < 0.001), all weights were recalculated for a random-effects solution (Gonzales-Barron et al., 2009). The relative size of the bubbles shown in Fig. 1 is proportional to the weight assigned to each primary study. In summary, this metaanalysis has shown how the body of information contained in all studies revealed a clearer picture of the state of knowledge. In Gonzales-Barron et al. (2009), this analysis was combined with a non-parametric bootstrap technique in order to build a stochastic regression between the Salmonella carriage rate in pigs entering the abattoirs and the resulting incidence on carcasses at the point of evisceration. The second case demonstrates the application of meta-analysis for the estimation of the overall effect of a critical process stage on the incidence of a pathogen under study. Specifically, the overall effect size of chilling on the recovery of Salmonella from pork carcasses was estimated. The chilling meta-analysis was conducted
on the RR measure (defined as the probability of encountering Salmonella-positive pig carcasses after chilling relative to the probability of encountering Salmonella-positive carcasses before chilling). The chilling meta-analysis confirmed on the grounds of increased statistical power the decreasing effect that chilling has on the recovery of Salmonella (U ¼ 27.3; p < 0.001). As there was no evidence of heterogeneity (Q ¼ 2.9; p ¼ 0.96), the fixed-effects meta-analysis was considered a suitable solution and it delivered a normal distribution of the overall effect of chilling (with a mean of w2.4 times reduction in Salmonella recovery on pig carcasses, Fig. 2). The parameterisation of relative risk was believed to be a more appropriate statistical measure for this meta-analysis because, being essentially a ratio of probability after chilling to probability before chilling, the different sensitivity values due to detection test, protocols and extent of swabs will be rather cancelled off in every primary study estimate, and this will therefore contribute to a better agreement among primary studies (since less sources of heterogeneity are affecting the effect size measure). This fact made the overall relative risk estimate of meta-analysis more generalisable, in the sense that the overall reduction factor of w2.4 in Salmonella prevalence can be extended to the broad range of tests that were used in the studies to determine Salmonella presence in pig carcasses. Because meta-analysis relies on actual data, the effect size outcome distribution can be used instead of, or in addition to, expert judgment in quantitative risk assessment models. Hence, it is expected that the normal distribution of the effect size of chilling on Salmonella prevalence will provide a more precise and realistic input distribution of the chilling stage for risk assessment models. Meta-analysis, if carefully constructed and implemented, can assist food safety researchers to determine the extent to which accumulated evidence tends to provisionally confirm or conclusively refute a specified theory of the intervention under investigation. The claimed strengths of meta-analysis are all contingent on the important condition that the measures that represent and share the same theoretical concepts within a meta-analysis be of at least satisfactory validity (related to the principle of ‘rubbish in e rubbish out’). The main limitations of meta-analysis are related to quality of primary research and publication bias (Noble, 2006). Meta-analysis is incapable of correcting the limitations imposed by underpowered primary studies that introduce small sample bias with overstatement of the estimated effect size as well as insensitivity to the influence of relevant moderators on outcomes. Underpowered
4.5
Precision (1/standard error )
Proportion of resulting Salmonellapositive eviscerated carcasses
0.40 0.35 0.30 0.25 0.20 0.15 0.10 2
R = 0.770
0.05 0.00
4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
0
0.2
0.4
0.6
0.8
Proportion of slaughter pigs carrrying Salmonella in caeca Fig. 1. Bubble plot of the cecal Salmonella carriage rate in slaughter pigs and the resulting Salmonella-positive eviscerated carcasses. Bubble area is proportional to the weight assigned to each primary study.
-2.5
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
Log relative risk (log p T /pR) Fig. 2. Funnel plot of the relative risk of the Salmonella recovery on pig carcasses after chilling (pT) in relation to before chilling (pR). Vertical line at log pT/pR ¼ 0.88 (pR/pT ¼ w2.4) represents the overall effect size, and marker area is proportional to the weight assigned to each primary study.
U. Gonzales-Barron, F. Butler / Food Microbiology 28 (2011) 823e827
primary research, that is, research that employs samples of insufficient size and insufficient variability, simultaneously constrains detections of small statistically-significant differences between intervened and control subjects, and increases the likelihood of accepting the null hypothesis as true when it is false (type II error). On the other hand, publication bias exists because research with statistically-significant results is potentially more likely to be published than work with non-significant results. The presence of publication bias in a meta-analysis can be assessed informally by inspection of a funnel plot, which plots the effect size of each study against some measure of its precision (1/standard error). The resulting plot should be shaped like a funnel if no publication bias is present. This shape is expected because trials of decreasing size have increasingly large variation in their effect size estimates due to random variation becoming increasingly influential. However, if the chance of publication is greater for larger trials or trials with statistically-significant results, some small non-significant studies may not appear in the literature, leading to the omission of trials in one corner of the plot. Fig. 2 shows a funnel plot of the primary studies used to conduct the chilling meta-analysis. Visual inspection would suggest that, despite the few studies utilized for this meta-analysis, there is little evidence of publication bias in this dataset. 4. Conclusions While meta-analysis is not without limitations (underpowered primary research and publication bias), its systematic approach comprising a broad range of techniques merits consideration among food safety researchers to integrate the current body of knowledge and data on targeted issues along the complex continuum of agro-food production, and to furnish increased credibility to findings in the field. As reviewed in the two metaanalysis applications, while primary studies might be weak or reach contradictory conclusions, the body of information contained in all studies might reveal a clearer picture of the state of knowledge, and can offer valuable quantitative information of effect size in the form of distributions that can be inserted into risk assessment models. Meta-analysis can highlight areas where there is insufficient
827
evidence of the efficacy of interventions, where there is absence of high quality studies, or where there are common methodological flaws in the available research, and can therefore provide direction for future research. Acknowledgments The authors wish to acknowledge Safe Food, The Food Safety Promotion Board and the Food Institutional Research Measure (FIRM) administered by the Irish Department of Agriculture, Fisheries and Food. References Bollaerts, K., Aerts, M., Faes, C., Grijspeerdt, K., Dewulf, J., Mintiens, K., 2008. Human salmonellosis: estimation of dose-illness from outbreak data. Risk Anal. 28 (2), 427e440. Glass, G.V., 1976. Primary, secondary and meta-analysis of research. Educ. Res. 5 (1, 3), 3e8. Gonzales-Barron, U., Bergin, D., Butler, F., 2008. A meta-analysis study of the effect of chilling on Salmonella prevalence on pork carcasses. J. Food Prot. 71 (7), 1330e1337. Gonzales-Barron, U., Soumpasis, I., Butler, F., Duggan, S., Prendergast, D., Duffy, G., 2009. Estimation of prevalence of Salmonella spp. on pig carcasses and pork joints using a quantitative risk assessment model aided by meta-analysis. J. Food Prot. 72 (2), 274e285. Noble, J.H., 2006. Meta-analysis: methods, strengths, weaknesses and political uses. J. Lab. Clin. Med. 147 (1), 7e20. Patil, S.R., Morales, R., Cates, S., Anderson, D., Kendall, D., 2004. An application of meta-analysis in food safety consumer research to evaluate consumer behaviours and practices. J. Food Prot. 67 (11), 2587e2595. Sánchez, J., Dohoo, I.R., Christensen, J., Rajic, A., 2007. Factor influencing the prevalence of Salmonella spp. in swine farms: a meta-analysis approach. Prev. Vet. Med. 81, 148e177. Sargeant, J.M., Amezcua, M., Rajic, A., Waddell, L., 2005. A Guide to Conducting Systematic Reviews in Agri-food Public Health. Available at:. Food Safety Research and Response Network, Canada http://www.fsrrn.net/UserFiles/File/ conductingsysreviewsenglish[1].pdf. Sargeant, J.M., Rajic, A., Read, S., Ohlsson, A., 2006. The process of systematic review and its application in agri-food public health. Prev. Vet. Med. 76, 141e151. Sutton, A.J., Abram, K.R., Jones, D.R., 2001. An illustrated guide to the methods of meta-analysis. J. Eval. Clin. Pract. 7 (2), 135e148. Vialette, M., Pinon, N., Leporq, B., Dervin, C., Membré, J.M., 2005. Meta-analysis of food safety information based on a combination of a relational database and a predictive modelling tool. Risk Anal. 25 (1), 75e83. Yates, F., Cochran, W.G., 1938. The analysis of groups of experiments. J. Agric. Sci. 28 (1), 556e580.