Microbial diversity of consumption milk during processing and storage

Microbial diversity of consumption milk during processing and storage

International Journal of Food Microbiology 266 (2018) 21–30 Contents lists available at ScienceDirect International Journal of Food Microbiology jou...

1MB Sizes 0 Downloads 54 Views

International Journal of Food Microbiology 266 (2018) 21–30

Contents lists available at ScienceDirect

International Journal of Food Microbiology journal homepage: www.elsevier.com/locate/ijfoodmicro

Microbial diversity of consumption milk during processing and storage a,⁎

b

a

T

b

Davide Porcellato , Marina Aspholm , Siv Borghild Skeie , Marte Monshaugen , Johanne Brendehaugc, Hilde Mellegårdb a b c

Faculty of Chemistry, Biotechnology and Food Science, The Norwegian University of Life Sciences, P.O. Box 5003, N-1432 Ås, Norway Faculty of Veterinary Medicine, The Norwegian University of Life Sciences, P.O. Box 5003, N-1432 Ås, Norway TINE SA, Måltidets Hus, Richard Johnsensgate 4, P.B. 8034, N-4068, Stavanger, Norway

A R T I C L E I N F O

A B S T R A C T

Keywords: Milk microbiota Milk storage Raw milk Bacillus cereus Milk quality

Bovine milk contains a complex microbial community that affects the quality and safety of the product. Detailed knowledge of this microbiota is, therefore, of importance for the dairy industry. In this study, the bacterial composition of consumption milk was assessed during different stages in the production line and throughout the storage in cartons by using culturing techniques and 16S rRNA marker gene sequencing. Monthly samples from two dairies were analyzed to capture the seasonal variations in the milk microbiota. Although there was a core microbiota present in milk samples from both dairies, the composition of the bacterial communities were significantly influenced by sampling month, processing stage and storage temperature. Overall, a higher abundance of operational taxonomic units (OTUs) within the order Bacillales was detected in samples of raw and pasteurized milk from the spring and summer months, while Pseudomonadales and Lactobacillales OTUs were predominant in the winter months. OTUs belonging to the order Lactobacillales, Pseudomonadales, Clostridiales and Bacillales were significantly more abundant in milk samples taken immediately after pasteurization compared to raw milk samples. During storage of milk in cartons at 4 °C, the bacterial composition remained stable throughout the product shelf life, while storage at 8 °C significantly increased the abundance of OTUs belonging to the genus Bacillus and the plate count levels of presumptive Bacillus cereus. The knowledge obtained in this work will be useful to the dairy industry during their quality assurance work and risk assessment practices.

1. Introduction Bovine milk is a nutritious food product consumed as a beverage or used as an ingredient in the production of a wide range of dairy and non-dairy products. Raw milk contains highly diverse bacterial populations, some of which such as lactic acid bacteria, are beneficial for milk processing, and others, such as spore-forming and psychrotrophic bacteria, are involved in spoilage and disease (as reviewed by Quigley et al., 2013a). Microbial contamination in the dairy chain may occur on the farm, during transport and in the processing facilities. Therefore, the composition of the raw and pasteurized milk microbiota is greatly influenced by several factors, such as farm management practices, seasons and hygienic practices and storage conditions throughout the value chain (Mallet et al., 2012; Quigley et al., 2013a; Vithanage et al., 2016). To avoid milk spoilage and ensure safe products, the dairy industry applies thermal treatment to reduce the microbial load. However, spore-formers and other heat-resistant bacteria may survive this treatment (Christiansen et al., 2006; Novak et al., 2005). The storage of



processed milk products at an optimal temperature is also important to minimize microbial growth and to reduce enzymatic activities, thereby maintaining the intended product shelf life. Previous studies have shown that the cold chain temperature is not always kept within the recommended range, particularly in consumer refrigerators, which increases the risk of spoilage and growth of potential pathogenic microorganisms in food products during storage (Rossvoll et al., 2014; Schmidt et al., 2012). To ensure delivery of high quality products with long shelf life, the dairy industry needs to evaluate milk quality throughout the value chain. Traditionally, culture-dependent techniques have been used to assess the milk microbiota, both in research and industry (Raats et al., 2011). However, these methods are usually time consuming and labor intensive and may not provide a realistic picture of the diversity of the microbial communities. Recent advances in genomic technologies and bioinformatics tools have enabled characterization of the total microbiota directly from food matrices (Ercolini, 2013). By using cultureindependent methods, such as high throughput sequencing (HTS), bacterial genera not previously associated with milk, e.g., Bacteroides,

Corresponding author. E-mail address: [email protected] (D. Porcellato).

https://doi.org/10.1016/j.ijfoodmicro.2017.11.004 Received 27 February 2017; Received in revised form 2 November 2017; Accepted 7 November 2017 Available online 08 November 2017 0168-1605/ © 2017 Elsevier B.V. All rights reserved.

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

colony counts at end of shelf life at 8 °C). The impact of freezing on the total aerobic and B. cereus colony counts was tested by plating previously frozen and fresh replicas of the same milk sample. This test revealed that freezing had no or very little effect on the colony counts. Bacterial levels in raw milk (from the total aerobic plate counts) were analyzed by two-way ANOVA model using the sampling month and dairy (A and B) as factors. Bacterial levels in pasteurized milk (total aerobic plate count and B. cereus colony count) were analyzed using a linear cross design model including dairy, sampling month and storage temperature as fixed factors and sampling day as a covariate. The model also included the interaction between temperature and the covariate. All statistical tests were performed with R statistic version 3.3.0 (R Core Team, 2017).

Parabacteroides and Faecalibacterium, have now been detected in cow milk (Quigley et al., 2013b). Also, while Pseudomonas spp. appeared to be eliminated by pasteurization as detected by plate counts, cultureindependent analyses of the same samples showed a reduction in bacterial levels rather than an elimination (Quigley et al., 2013b). Knowledge of the actual composition of the bacterial community in milk throughout the value chain will facilitate the dairy industry in their production of safe and high-quality products in a long-term sustainable manner. In Norway, a large-scale study on contamination routes in the dairy industry was performed in 1995–1998 using plate count techniques (Eneroth et al., 1998; Svensson et al., 1999). However, no systematic survey on the composition of the milk microbiota throughout the Norwegian dairy value chain has been conducted in recent years. In the current work, the microbial composition of milk samples from two dairies was characterized. Samples were taken from raw milk in silo tanks through processing at the dairy plant, and during storage at 4 °C and 8 °C until end of product shelf life. In addition, total aerobic bacterial counts and presumptive Bacillus cereus levels (hereafter designated B. cereus), which are commonly used microbiological parameters in industry today, were determined by traditional culturing techniques.

2.3. 16S marker gene sequencing and bioinformatics analysis The microbial composition of all milk samples (n = 864) was studied according to the method described by Porcellato et al. (2016), with minor changes. In brief, the variable region V3 and V4 of the bacterial 16S rRNA gene was amplified using the universal primers Uni340F (5′CCTACGGGRBGCASCAG-3′) and Bac806R (5′-GGACTACYVGGGTATCTAAT-3′) (Takai and Horikoshi, 2000). The PCR reaction was performed using 4 μL of DNA. 20 μL of the resulting PCR product was purified using 0.8 × of Agencourt AMPure XP beads (Beckman Coulter Inc., Brea, CA, USA) according to the manufacturer's instructions. Five μL of the purified PCR product was used as template for the second PCR using customized primers with unique sample barcodes (Table S1). Library normalization was performed using the SequalPrep™ Normalization plate (Thermo Fischer Scientific, Oslo, Norway) and quantified using the PerfeCTa NGS quantification kit (Quanta Biosciences, Beverly, MA, USA). Sequencing of the library was done using the Illumina Miseq platform (Illumina, San Diego, CA, USA) with a 300 bp paired-end sequencing kit. Raw Illumina fastq files were merged and quality filtered using the Usearch 8.0 (Edgar, 2010). Only sequences with lengths over 380 base pairs were kept for further analysis. The resulting files were demultiplexed using Qiime 1.9.0 (Caporaso et al., 2010). The Usearch operational taxonomic unit (OTU) picking algorithm (Edgar, 2013) with clustering at 97% pairwise identity was chosen as OTU picking method. Singletons and chimeras sequences were removed before the clustering step as part of the Usearch pipeline. The taxonomy was assigned to each OTU as previously described (Porcellato and Skeie, 2016). Sequences were deposited in the European Nucleotide Archive (ENA) under the accession number PRJEB23297.

2. Materials and methods 2.1. Milk sampling and DNA extraction The bovine milk samples (n = 864) included in this experiment were collected during a 13-month period (from June 2015 to June 2016) from two dairies (A and B) situated in two different geographical locations in Norway. Dairy A is a large-scale processing facility (annual production > 80 million L of milk), while Dairy B produces substantially less milk (annual production < 40 million L of milk). Each month, with the exception of December 2015, the following samples were collected successively from each dairy: raw bovine milk from silo tanks (100 mL), homogenized and pasteurized (72 °C for at least 15 s) full-fat milk from intermediate storage tanks (100 mL), and 15 one-liter cartons produced the same day. Milk from the same silo and intermediate storage tanks, where the temperature was kept at 3–4 °C, were sampled throughout the trial period. During transport to the laboratory, the milk samples were kept cold (maximum 4 °C) and the transport time did not exceed 1.5 h. The samples were held at 4 °C at the laboratory until the next day, when culturing and DNA extraction took place. Samples from the silo tanks, the intermediate storage tanks and three of the pasteurized milk cartons were analyzed at Day 1 after sampling, while the remaining cartons were kept at 4 °C or 8 °C for 6–8 days (midshelf life) or 13–14 days (end of shelf life reported on the milk cartons) before processing. Three replicates of each sample from each silo tank and intermediate storage tank were analyzed, while three cartons, with two replicate samples from each, were analyzed for every temperature and time point. DNA was extracted from 10 mL of milk from each sample as previously described by Porcellato et al. (2016) and stored at − 20 °C until further analysis. For all samples analyzed, milk samples containing 15% glycerol were stored at − 80 °C as back-up material.

2.4. Statistical analysis of the microbial community The final OTU table was normalized using the cumulative-sum scaling method to account for different sequencing depth (Paulson et al., 2013). This method has previously been shown to perform similarly or better compared to other methods of normalization (Kable et al., 2016; Paulson et al., 2013). Alpha diversity (within-sample diversity) was calculated with the R-package Vegan (Oksanen et al., 2017). Richness and diversity indexes were calculated with the Chao1 and Shannon index, respectively. Beta diversity (between-sample diversity) was estimated using both the weighted and unweighted Unifrac distance between the milk samples. Nonmetric dimensional scaling (NMDS) was computed from the weighted Unifrac distance matrix using the R-package Vegan. The weighted Unifrac distance matrix was also used to compute the permutational multivariate analysis of variance (Adonis function in R-package Vegan) using the different experimental factors (dairy plant, month, temperature, day and sample type). Differential abundances for each OTU between sample groups were determined using the zero-inflated log-normal mixture model (available in the metagenomeSeq R package, http://bioconductor.org/packages/ release/bioc/html/metagenomeSeq.html) after filtering the OTU table

2.2. Culture conditions The total aerobic plate counts were determined for all milk samples on Plate Count Skimmed Milk Agar (Merck KGaA, Darmstadt, Germany), according to ISO 4833-1:2013 (Anon., 2013). Enumeration of viable B. cereus isolates from milk cartons was performed on B. cereus selective agar with supplements according to NMKL method No. 67 (Nordic Committee on Food Analysis), but without additional plating on blood agar plates. All culture media were obtained from Oxoid Ltd., Hampshire, England. Frozen back-up samples were used for bacterial enumeration in two samplings from Dairy B, July 2015 (total aerobic colony counts at end of shelf life at 4 °C) and February 2016 (B. cereus 22

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

Fig. 1. Total aerobic colony counts of milk samples collected from Dairy A (A) and Dairy B (B), expressed as arithmetic means ± standard deviations (SD) of the logtransformed data (SDs are not given when only 2 (out of 3) valid colony counting's were recorded).

occasion, a contamination problem during processing was reported from the dairy plant. Samples from full-fat milk cartons stored until end of shelf life at 4 °C and 8 °C were analyzed for B. cereus. The B. cereus counts were not influenced by dairy or sampling month. However, both temperature and storage time, in addition to the interaction between these two factors, significantly influenced the B. cereus counts. When stored at 4 °C, the milk samples showed very low or no detectable B. cereus levels (detection limit of 1 cfu/mL) from Day 1 until end of the storage period (Fig. 2). In contrast, storage at 8 °C resulted in increased B. cereus levels at mid shelf life and at the end of shelf life compared to storage at 4 °C. At the end of shelf life at 8 °C, the level of B. cereus was 5.28 ± 2.16 (Fig. 2).

by keeping features present in more than half of the samples considered and with minimum sequencing depth of 2000 sequences per sample. The raw milk core microbiota was defined by including all taxa present in all samples examined. A rarefaction curve plot was constructed for 72 out of 864 milk samples after rarefaction of the sequences to a depth of 3000 sequences per sample. 3. Results 3.1. Bacterial plate counts In total, 864 milk samples were collected over a 13-month period and cultured on plate count skimmed milk agar. The results for the total aerobic plate counts are shown in Fig. 1. The bacterial level in raw milk ranged from 3.61 to 5.5 log cfu/mL (mean 4.25, median 4.12). In raw milk, the bacterial levels were not significantly different between the dairies, while a significant effect of the month on the bacteria level was detected (p < 0.05). From the raw milk samples collected in the silo tanks to the pasteurized milk collected from the intermediate storage tanks, the total aerobic plate counts were reduced by 1.1 ± 0.3 (Dairy A) and 1.2 ± 0.5 log cfu/mL (Dairy B; Fig. 1). After packaging and during storage of milk cartons, the temperature, the storage time, and the interaction between the storage time and the temperature significantly affected the bacterial levels (p < 0.05). However, both the sampling month and the dairy factor had no effect on the bacterial levels. Storage time was decisive for the total aerobic plate counts during storage at 8 °C (Fig. 1). In particular, at the end of storage at 8 °C, the total bacterial counts were 2.81 ± 0.99 log cfu/mL higher than in milk samples stored at 4 °C (Fig. 1, Fig. S1). Similar bacterial levels in cartons stored at 4 °C and 8 °C were only observed during two sampling months (April 2016 and July 2015 from Dairy A). On the latter

3.2. The composition of the milk microbiota All milk samples were subjected to 16S rRNA marker gene sequencing. The average number of sequences per sample was 15,105 (median: 12,561) and the total number of OTUs detected and assigned to the Bacteria kingdom was 3380. As the number of samples increased, the rarefaction curves of OTUs gradually flattened out indicating that much of the diversity within the samples was recovered (Fig. S2). Multivariate analysis, using the non-metric multidimensional scaling (NMDS) of the weighted Unifrac distance matrix, showed that all samples of raw and pasteurized milk from dairy silo tanks and intermediate storage tanks obtained a good (stress value: 0.044) and linear fit (R2) between the observed distance and the ordination distance of 0.992. There were no significant differences in the microbial composition between the two dairies for both raw (p = 0.568) and pasteurized milk (p = 0.233); therefore, the results from the two dairies are reported together in the following text unless otherwise stated. The 23

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

Fig. 2. Presumptive Bacillus cereus colony counts for milk samples collected from Dairy A (A) and Dairy B (B), expressed as arithmetic means ± SDs of the log-transformed data (n + 1).

taxa of the core microbiota for both dairies were Pseudomonas, Bacillus, Acinetobacter, Streptococcus, Lactococcus, Chryseobacterium and uncultured Firmicutes. The low abundance OTUs (OTUs with < 1% of total reads) constituted a substantial part of the milk microbiota in some samples, where these accounted for > 30% of the total reads (Fig. 4). Differential abundance analysis was used to identify OTUs that differed between the raw milk and the pasteurized milk samples. In total, 21 order-level OTUs were significantly more abundant in raw milk compared to pasteurized milk (Fig. 3B). Of these, 15 OTUs belonged to the order Clostridiales. On the other hand, 27 OTUs were significantly more abundant in pasteurized milk compared to raw milk. Ten of these OTUs belonged to the order Lactobacillales and six to each of the orders Clostridiales and Pseudomonadales (Fig. 3B).

microbial compositions of milk samples collected from the silo tanks before pasteurization and from the intermediate storage tanks after pasteurization were significantly (Adonis p < 0.001) influenced by the sampling month and by the sample type (raw milk and pasteurized milk, p < 0.001). The NMDS plots also showed a clear separation of samples from different seasons and between samples of raw and pasteurized milk (Fig. 3A). Taxonomic assignment of OTUs at the genus level showed that Pseudomonas and Bacillus were most abundant in samples of raw and pasteurized milk from both dairies, accounting for 26% and 28% of the total sample sequences, respectively (Fig. 4). The number of observed OTUs, depicted as alpha diversity richness (Fig. S3), increased in samples collected during the late autumn and winter months compared to other periods of the year. Pseudomonas was generally the most abundant OTU in the late autumn and winter months (from September to March), while Bacillus OTUs dominated the microbiota in the spring and summer months. In raw milk samples a core microbiota was detected, which was consistent throughout the entire sampling period (Table S2). This core microbiota was detected in all the samples analyzed and consisted of 20 genera, of which 13 were common to both dairies, while three and four genera were specific for Dairies A and B, respectively. The core genera accounted for 74 and 83% of the total reads assigned to raw milk samples from Dairy A and Dairy B, respectively, while the dairy plant specific genera constituted only a minor part of the total microbiota (< 1% of the reads). The most abundant

3.3. The composition of the bacterial microbiota of carton milk during storage at different temperatures The composition of the bacterial microbiota of milk from both dairies was assessed during storage at 4 °C and 8 °C. The microbial composition was not significantly different between the two dairies for the milk cartons samples stored at 4 °C (p = 0.356) and 8 °C (p = 0.682) until the end of shelf life. Samples kept at 4 °C did not demonstrate any significant changes in the bacterial composition between the sampling points at Day 1 and at the end of shelf life (Adonis 24

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

genus Bacillus (Fig. 6). During the spring and summer months (April to September), there was a higher abundance of Bacillus OTUs in mid-shelf life samples compared to other times of the year (Adonis p < 0.001). Milk cartons stored at 4 °C demonstrated a higher occurrence of Bacillus OTUs from April to June 2016 compared to the rest of the sampling period. However, this variation was not as evident as for milk cartons stored at 8 °C. The seasonal differences in Bacillus spp. abundance detected throughout the milk carton shelf life were paralleled by seasonal variations in Bacillus spp. levels in the raw milk samples. Raw milk with high levels of Bacillus resulted in carton milk with high levels of Bacillus at the end of the storage period at both temperatures (Fig. 4A, Fig. 4C and Fig. 6A–B). 4. Discussion In this study, the bacterial counts and the composition of the microbiota of raw milk, pasteurized full-fat milk and pasteurized carton milk were determined over a 13-month period. The influence of storage time and temperature on the milk microbiota was also addressed. The total number of OTUs detected over the entire experimental period was over 3000, and in some samples of raw milk the estimated number of OTUs was as high as 1000 per sample. The low abundance OTUs constituted a substantial proportion (26% of the total number of reads) of the total microbiota of raw milk in the present study, which is in accordance with other reports (Kable et al., 2016; Quigley et al., 2013a). The high bacterial diversity confirms that the milk microbiota is a very complex biological community. Possible sources of bacterial contamination in the dairy value chain are indeed many: from the animal itself, the farm environment, transportation, storage and processing at the dairy plant (Huck et al., 2008; Oliver et al., 2005). The microbial composition is also influenced by different events along the value chain. For example, a previous study showed that there was a shift in dominance from a Gram-positive to a Gram-negative bacterial flora from farm milk to dairy silo tank milk (Fricker et al., 2011). There was also a greater bacterial diversity in the farm milk compared to milk from the silo tank. In addition, it has been reported that transfer of milk from tanker trucks to dairy storage silos can lead to significant changes in certain bacterial populations, including taxa defined as part of the core microbiota of raw milk (Kable et al., 2016). In the present work, no significant differences in bacterial levels or microbial composition were detected between the two dairies involved. However, for both dairies significant differences between sampling months, processing stages and storage temperatures were found. The dairy A and B were included in the study because of differences in production volumes. Both dairies are located in adjacent regions of Norway, with similar climatic conditions. However, there is no overlap in the milk collection routes between the two dairies. Differences in soil, herd management and dairy plant management might impact the milk microbiota and thereby the raw milk quality, however, these factors did not contribute to any significant differences between dairy A and dairy B in the present study. In the milk samples, a higher alpha diversity was estimated in raw and pasteurized milk during the cold season compared to the spring and summer months. This differs from findings in an American study, where the bacterial species richness of raw milk from tanker trucks was highest during spring sampling (Kable et al., 2016). However, factors such as farm management, weather conditions, sanitation practices and type of samples analyzed (tanker trunks versus silo tanks) may explain this difference. The tendency of lower alpha diversity, calculated using the Chao1 index, in raw and pasteurized milk during the warm season, as observed in this study, might also be explained by the relative increase in the abundance of the genus Bacillus during this time of the year. On the contrary, the relative proportion of Bacillus decreased in the same sample categories during the late autumn and winter months, while other OTUs, including the genera of Pseudomonas, Streptococcus and Acinetobacter, dominated during this period. One consideration

Fig. 3. A) Non-metric multidimensional scaling analysis plots of the weighted Unifrac distance matrix for samples of raw and pasteurized milk obtained from silo tanks and intermediate storage tanks, respectively. The ellipses show the standard deviations of the weighted average of scores grouped by month with 95% confidential interval. Results from both plants are merged in the figure. B) Differential abundance plot showing orderlevel OTUs significantly more abundant in raw or pasteurized milk. Negative values show the OTUs significantly more abundant in raw milk samples, while positive values show the OTUs significantly more abundant in pasteurized milk samples. Each column represents an OUT within the order described by the color. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

p = 0.265; Figs. 5 and 6). In contrast, at 8 °C, the bacterial composition changed significantly throughout the storage period (Adonis p < 0.001). In the NMDS plot (Fig. 5A), there was a clear separation between samples stored at 4 °C and 8 °C. Differential abundance analysis revealed that storage at 8 °C significantly (p < 0.05) increased OTUs within the orders Bacillales (22 OTUs) and Lactobacillales (8 OTUs) compared to storage at 4 °C, where only 2 and 5 OTUs, respectively, belonged to these orders (Fig. 5B). OTUs belonging to the order Clostridiales were also more abundant during storage at 4 °C compared to 8 °C. The genus-level OTUs that were most abundant in samples stored at 8 °C were Bacillus (n = 14 OTUs), followed by Paenibacillus (n = 3 OTUs), Solibacillus (n = 2 OTUs), Anoxybacillus, Geobacillus and Jeotgalicococcus (n = 1 OTU each). At mid-shelf life and end of shelf life, all milk samples from cartons stored at 8 °C contained a high proportion of OTUs belonging to the 25

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

Fig. 4. Relative abundance plots of the main bacterial OTUs for samples of raw milk and pasteurized milk collected from Dairy plant A and Dairy plant B in the period from June 2015 (Jun15) to June 2016 (Jun16). The legend shows the taxonomic assignment of the OTUs at genus level (F: OTUs clustered at family level). For each month all replicate samples are shown.

lipolytic enzymatic activities (De Jonghe et al., 2011; Quigley et al., 2013a; Quigley et al., 2013b; Raats et al., 2011; Teh et al., 2011). These degradation activities have been shown to be particularly problematic in winter milk (Marchand et al., 2009), and may be explained by increased proportions of Pseudomonas spp. in the milk microbiota during this season, as observed in the present study. Streptococcus spp. was detected in all samples of raw milk in this work, and was also defined as part of the core microbiota of raw milk in the study by Kable et al. (2016). Thermophilic Streptococcal bacteria are able to survive the pasteurization step and produce a biofilm in dairy equipment (Delgado et al., 2013; Marchand et al., 2012). Certain streptococcal species are also known to cause mastitis and may therefore be present in raw milk (Kuehn et al., 2013; Oikonomou et al., 2014; Osteras et al., 2006). This and other studies report the presence of the genus Acinetobacter in raw milk (Kable et al., 2016; Quigley et al., 2013a; Quigley et al., 2013b; Rasolofo et al., 2010; Vaz-Moreira et al., 2011). Many members of this genus are widely distributed in nature; they have been detected in biofilms on different surfaces and have been associated with

while interpreting these results is that the increase in abundance of one OTU has a direct impact on the relative abundance of the other OTUs. Thus, when one OTU increases in a sample, certain low abundance OTUs might fall under the detection limit since they are constrained by the total number of sequences. However, this seasonal variation in relative levels of Bacillus spp. is in accordance with previous studies where higher plate count levels of B. cereus were detected during the summer period compared to other times of the year (Bartoszewicz et al., 2008; Larsen and Jorgensen, 1997). Although a variation in the bacterial composition was observed over the 13 months sampling period, several taxa were detected in the raw milk samples over the year and were, therefore, defined as the core microbiota of raw milk. Most of these taxa, including Bacillus, Pseudomonas, Streptococcus, Lactococcus, and Acinetobacter, have also been categorized as part of the milk core microbiota in previous studies (Kable et al., 2016; Quigley et al., 2013b). Spoilage of raw milk has mainly been attributed to the presence of psychrotrophic species within the genus Pseudomonas, with particular emphasis on proteolytic and 26

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

connected to these viability assays, especially when applied on environmental samples, as reviewed by Fittipaldi et al. (2012). Lower PCR signals by damaged cells in complex microbiological samples compared to pure cultures, which is highly relevant for the pasteurized milk samples in this study, is one of these limitations. In addition, there are practical implications such as the preference of adding the dyes prior to sample freezing to avoid biases due to cell impairment. In both raw and pasteurized milk samples from the present study, dead cells are likely to be present. However, as Kable et al. (2016) also assumed, dead cells are likely to constitute only a minor part of the total bacterial microbiota in raw milk, as these milk samples have not undergone pasteurization. Post pasteurization milk is, on the other hand, likely to contain a high proportion of dead cells, and this is important to take into account when assessing the 16S sequencing results from this work. OTUs within the orders of Lactobacillales, Pseudomonadales, Clostridiales and Bacillales were significantly more abundant in the intermediate storage tanks after pasteurization compared to raw milk from the storage silos. The increase in certain OTUs may relate to survival of certain species while others are destroyed, which appears as a relative increased abundance of taxa that survive the pasteurization process. Spore-forming bacteria within the orders Clostridiales and Bacillales (Buehner et al., 2014; Granum and Lindbäck, 2013; Jay et al., 2005) and several thermoduric species within the order Lactobacillales (De Angelis et al., 2004) are known to survive the high temperature/ short time (HTST) pasteurization. Pseudomonas species, as referred to in the introduction, are also able to survive pasteurization in a non-culturable form (Gunasekera et al., 2002; Quigley et al., 2013b). This increase in the relative abundance of certain OTUs could also reflect bacterial contamination during processing. Bacteria surviving pasteurization may form a biofilm in the production equipment and recontaminate the product at different stages along the production line (Eneroth et al., 1998; Eneroth et al., 2001; Marchand et al., 2012; Svensson et al., 2000). Within all four taxa that were significantly more abundant in pasteurized milk (Lactobacillales, Pseudomonadales, Clostridiales and Bacillales), there are potential spoilage organisms, which may impact the product quality during storage (Jay et al., 2005; Quigley et al., 2013a). In addition to evaluating the raw and pasteurized milk microbiota, the bacterial levels and composition during storage at two different temperatures were investigated. Full-fat milk cartons from both dairies were kept at 4 °C and 8 °C until end of shelf life. These two storage temperatures were chosen to take both optimal and suboptimal refrigeration temperatures into account. The least controllable part of the cold chain of many foods from production to consumption is the endconsumer. The mean temperature of domestic refrigerators are in several studies found to be around 6 °C, but with substantially higher maximum registered temperatures, as summarized in a review article (James et al., 2008). There are not many studies conducted on Norwegian domestic refrigeration temperatures. However, a recent study of 46 household refrigerators found that the mean night temperature was 5.6 °C (range 1.3 to 9.9 °C) (Rossvoll et al., 2014). The total colony counts and B. cereus counts were several logs higher at the end of the storage period in samples kept at 8 °C compared to samples kept at 4 °C. The B. cereus counts showed the same trend as the 16S sequencing results, where the highest storage temperature resulted in a higher abundance of OTUs belonging to the genus Bacillus. This increase in Bacillus spp. was observed in samples from both the large and the small dairy plant. The higher relative levels of Bacillus spp. in samples stored at 8 °C also correlated with the relative amount of Bacillus spp. in raw milk. Bacillus spp. spores will survive HTST pasteurization and multiply following germination if environmental conditions allow it (Granum and Lindbäck, 2013; Jay et al., 2005). Some members of the B. cereus group are known to produce biofilms in storage and piping systems and, thereby, constitute a risk of recontamination of milk after pasteurization (Eneroth et al., 2001; Flint et al., 1997; Svensson et al., 2000). Certain species within this group are also pyschrotrophic and may grow

Fig. 5. A) Non-metric multidimensional scaling plots for milk samples stored for one day at 4 °C and to the end of shelf life at 4 °C and 8 °C. Ellipses indicate the standard deviations of the weighted average of scores for the three different groups with 95% confidential interval. Results from both dairy plants are merged in the same figure. B) Differential abundance plot showing order-level OTUs significantly more abundant in milk samples kept at 4 °C (negative values) and 8 °C (positive values). Each column represents an OTU within the order described by the color. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

contamination and spoilage of food products, including milk (Doughari et al., 2011). The total aerobic bacterial counts in the raw milk (silo tanks) in this study are in accordance with numbers reported elsewhere (Fricker et al., 2011; Hayes et al., 2001). During storage at 4 °C until the end of shelf life, the cfu levels diverged only slightly from the level detected at Day 1 after production. On only one occasion (Dairy B, July 2015, end of shelf life at 4 °C), a notably higher colony count was detected (> 6 log cfu/mL). This high level in Dairy B was attributed to a contamination problem on a filling machine, as reported by the dairy. According to the results from the present study, the microbial compositions of raw and pasteurized milk were significantly different. However, it is important to take into account that the sequencing method used in the present study did not differentiate between live and dead bacteria. In order to make this distinction in molecular studies, photoreactive and cell membrane impermeant DNA-binding dyes (such as propidium monoazide and ethidium monoazide) have been applied to remove PCR signals from dead cells (Porcellato et al., 2015; Soejima et al., 2012; Wang et al., 2014). However, there are several drawbacks 27

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

Fig. 6. Relative abundance plots for samples taken from milk cartons at Day 1 (stored at 4 °C; Day1), mid-shelf life (Mid SL) and end of shelf life (End SL) stored at 4 and 8 °C and collected from A) Dairy A or from B) Dairy B, in the period from June 2015 (Jun15) to June 2016 (Jun16). The percent abundance represents an average of the replicates. The legend shows the taxonomic assignment of the OTUs at genus level (F: OTUs clustered at family level).

milk stored at 8 °C was detected, showing that raw milk quality is essential to product quality under suboptimal storage conditions. Storage at 8 °C resulted in a significant increase in the levels of Bacillus spp. compared to storage at 4 °C. This highlights the importance of storing milk products at appropriate refrigeration temperatures after processing. However, Bacillus spp. and their subtypes vary in their ability to produce spoilage components and pathogenic substances. Therefore, further characterization of the Bacillus spp., present throughout the year and at different storage temperatures, is warranted. There is also a need for increased knowledge on how complex microbial communities, commonly found in milk, influence their growth, production of spoilage components and pathogenic substances. This knowledge is required to take additional steps to obtain dairy products with longer shelf life, less spoilage, less waste and to minimize the risk of foodborne disease. Supplementary data to this article can be found online at https:// doi.org/10.1016/j.ijfoodmicro.2017.11.004.

at temperatures below 10 °C, even as low as 4 °C (Christiansson et al., 1989; Stenfors and Granum, 2001; Vannetten et al., 1990). Strains growing at low temperatures, and designated presumptive B. cereus according to standard culturing methods, might, in fact, be B. weihenstephanensis, a psychrotolerant relative of B. cereus (Francis et al., 1998; Lechner et al., 1998), as culturing on B. cereus selective media does not distinguish between these two species. Several Bacillus species, especially within the B. cereus group (B. cereus sensu lato), are associated with food spoilage and toxicity (Arnesen et al., 2007; De Jonghe et al., 2010; Thorsen et al., 2006). High levels of B. cereus, which were detected in milk cartons stored at 8 °C in the present study, might, therefore, pose a quality and safety concern. However, this temperature is several degrees centigrade higher than the recommended domestic refrigeration temperature in Norway and, therefore, represents a suboptimal storage condition. In conclusion, this report presents a comprehensive study of the bacterial content of bovine milk, from two different Norwegian dairyprocessing plants, throughout a year. Despite the presence of a core microbiota in raw milk samples, the bacterial community present in the milk was found to be very dynamic and varied with time of the year, processing stage and storage conditions. However, no significant differences in colony counts or microbiota composition were detected between the two dairies differing in production volumes. Most notably, the genera Pseudomonas, Streptococcus and Acinetobacter were predominant in the winter months and Bacillus spp. in the summer months. A correlation between the levels of Bacillus spp. in raw milk and carton

Acknowledgements The authors would like to acknowledge Ahmed Abdelghani (Norwegian University of Life Sciences, NMBU) and the Norwegian dairy cooperative TINE SA for their assistance during sampling and DNA isolation. The authors would also like to thank Trygve Almøy for support during the statistical analysis. This work received financial contributions from the Norwegian Foundation for Research Levy on Agricultural Products (FFL) and the Norwegian Agricultural Agreement 28

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

Research Fund (JA) (grant no 244149/E50). Department of Chemistry, Biotechnology and Food Science (NMBU), Center for Food Safety (NMBU), and TINE SA also contributed economically to this project.

the impact of transfer to a milk processing facility. MBio 7. Kuehn, J.S., Gorden, P.J., Munro, D., Rong, R.C., Dong, Q.F., Plummer, P.J., Wang, C., Phillips, G.J., 2013. Bacterial community profiling of milk samples as a means to understand culture-negative bovine clinical mastitis. PLoS One 8. Larsen, H.D., Jorgensen, K., 1997. The occurrence of Bacillus cereus in Danish pasteurized milk. Int. J. Food Microbiol. 34, 179–186. Lechner, S., Mayr, R., Francis, K.P., Pruss, B.M., Kaplan, T., Wiessner-Gunkel, E., Stewartz, G.S.A.B., Scherer, S., 1998. Bacillus weihenstephanensis sp. nov. is a new psychrotolerant species of the Bacillus cereus group. Int. J. Syst. Bacteriol. 48, 1373–1382. Mallet, A., Gueguen, M., Kauffmann, F., Chesneau, C., Sesboue, A., Desmasures, N., 2012. Quantitative and qualitative microbial analysis of raw milk reveals substantial diversity influenced by herd management practices. Int. Dairy J. 27, 13–21. Marchand, S., Heylen, K., Messens, W., Coudijzer, K., De Vos, P., Dewettinck, K., Herman, L., De Block, J., Heyndrickx, M., 2009. Seasonal influence on heat-resistant proteolytic capacity of Pseudomonas lundensis and Pseudomonas fragi, predominant milk spoilers isolated from Belgian raw milk samples. Environ. Microbiol. 11, 467–482. Marchand, S., De Block, J., De Jonghe, V., Coorevits, A., Heyndrickx, M., Herman, L., 2012. Biofilm formation in milk production and processing environments; influence on milk quality and safety. Compr. Rev. Food Sci. Food Saf. 11, 133–147. Novak, J.S., Call, J., Tomasula, P., Luchansky, J.B., 2005. An assessment of pasteurization treatment of water, media, and milk with respect to Bacillus spores. J. Food Prot. 68, 751–757. Oikonomou, G., Bicalho, M.L., Meira, E., Rossi, R.E., Foditsch, C., Machado, V.S., Teixeira, A.G.V., Santisteban, C., Schukken, Y.H., Bicalho, R.C., 2014. Microbiota of cow's milk; distinguishing healthy, sub-clinically and clinically diseased quarters. PLoS One 9. Oksanen, J., Blanchet, F.G., Friendly, F., Kindt, R., Legendre, P., McGlinn, D., Minchin, P.R., O'Hara, R.B., Simpson, G.L., Solymos, P., Stevens, M.H.H., Szoecs, E., Wagner, H., 2017. Vegan: community ecology package. R package version 2.4-2. https:// CRAN.R-project.org/package=vegan. Oliver, S.P., Jayarao, B.M., Almeida, R.A., 2005. Foodborne pathogens in milk and the dairy farm environment: food safety and public health implications. Foodborne Pathog. Dis. 2, 115–129. Osteras, O., Solverod, L., Reksen, O., 2006. Milk culture results in a large Norwegian survey-effects of season, parity, days in milk, resistance, and clustering. J. Dairy Sci. 89, 1010–1023. Paulson, J.N., Stine, O.C., Bravo, H.C., Pop, M., 2013. Differential abundance analysis for microbial marker-gene surveys. Nat. Methods 10 (1200-+). Porcellato, D., Skeie, S.B., 2016. Bacterial dynamics and functional analysis of microbial metagenomes during ripening of Dutch-type cheese. Int. Dairy J. 61, 182–188. Porcellato, D., Magri, M., Narvhus, J., 2015. Viable cells differentiation improves microbial dynamics study of fermented milks. Int. Dairy J. 47, 136–142. Porcellato, D., Narvhus, J., Skeie, S.B., 2016. Detection and quantification of Bacillus cereus group in milk by droplet digital PCR. J. Microbiol. Methods 127, 1–6. Quigley, L., O'Sullivan, O., Stanton, C., Beresford, T.P., Ross, R.P., Fitzgerald, G.F., Cotter, P.D., 2013a. The complex microbiota of raw milk. FEMS Microbiol. Rev. 37, 664–698. Quigley, L., McCarthy, R., O'Sullivan, O., Beresford, T.P., Fitzgerald, G.F., Ross, R.P., Stanton, C., Cotter, P.D., 2013b. The microbial content of raw and pasteurized cow milk as determined by molecular approaches. J. Dairy Sci. 96, 4928–4937. R Core Team, R.C., 2017. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria(URL https://http://www.Rproject.org/). Raats, D., Offek, M., Minz, D., Halpern, M., 2011. Molecular analysis of bacterial communities in raw cow milk and the impact of refrigeration on its structure and dynamics. Food Microbiol. 28, 465–471. Rasolofo, E.A., St-Gelais, D., LaPointe, G., Roy, D., 2010. Molecular analysis of bacterial population structure and dynamics during cold storage of untreated and treated milk. Int. J. Food Microbiol. 138, 108–118. Rossvoll, E., Ronning, H.T., Granum, P.E., Moretro, T., Hjerpekjon, M.R., Langsrud, S., 2014. Toxin production and growth of pathogens subjected to temperature fluctuations simulating consumer handling of cold cuts. Int. J. Food Microbiol. 185, 82–92. Schmidt, V.S.J., Kaufmann, V., Kulozik, U., Scherer, S., Wenning, M., 2012. Microbial biodiversity, quality and shelf life of microfiltered and pasteurized extended shelf life (ESL) milk from Germany, Austria and Switzerland. Int. J. Food Microbiol. 154, 1–9. Soejima, T., Minami, J., Iwatsuki, K., 2012. Rapid propidium monoazide PCR assay for the exclusive detection of viable Enterobacteriaceae cells in pasteurized milk. J. Dairy Sci. 95, 3634–3642. Stenfors, L.P., Granum, P.E., 2001. Psychrotolerant species from the Bacillus cereus group are not necessarily Bacillus weihenstephanensis. FEMS Microbiol. Lett. 197, 223–228. Svensson, B., Eneroth, A., Brendehaug, J., Christiansson, A., 1999. Investigation of Bacillus cereus contamination sites in a dairy plant with RAPD-PCR. Int. Dairy J. 9, 903–912. Svensson, B., Eneroth, A., Brendehaug, J., Molin, G., Christiansson, A., 2000. Involvement of a pasteurizer in the contamination of milk by Bacillus cereus in a commercial dairy plant. J. Dairy Res. 67, 455–460. Takai, K., Horikoshi, K., 2000. Rapid detection and quantification of members of the archaeal community by quantitative PCR using fluorogenic probes. Appl. Environ. Microbiol. 66 (5066 −+). Teh, K.H., Flint, S., Palmer, J., Lindsay, D., Andrewes, P., Bremer, P., 2011. Thermoresistant enzyme-producing bacteria isolated from the internal surfaces of raw milk tankers. Int. Dairy J. 21, 742–747. Thorsen, L., Hansen, B.M., Nielsen, K.F., Hendriksen, N.B., Phipps, R.K., Budde, B.B., 2006. Characterization of emetic Bacillus weihenstephanensis, a new cereulide-producing bacterium. Appl. Environ. Microbiol. 72, 5118–5121.

References Anon., 2013. ISO 4833-1:2013. Microbiology of the food chain. Horizontal method for the enumeration of microorganisms. In: Part 1: Colony Counts at 30 Degrees C by the Pour Plate Technique. Arnesen, L.P.S., O'Sullivan, K., Granum, P.E., 2007. Food poisoning potential of Bacillus cereus strains from Norwegian dairies. Int. J. Food Microbiol. 116, 292–296. Bartoszewicz, M., Hansen, B.M., Swiecicka, I., 2008. The members of the Bacillus cereus group are commonly present contaminants of fresh and heat-treated milk. Food Microbiol. 25, 588–596. Buehner, K.P., Anand, S., Djira, G.D., Garcia, A., 2014. Prevalence of thermoduric bacteria and spores on 10 Midwest dairy farms (vol 97, pg 6777, 2014). J. Dairy Sci. 97, 8009–8016. Caporaso, J.G., Kuczynski, J., Stombaugh, J., Bittinger, K., Bushman, F.D., Costello, E.K., Fierer, N., Pena, A.G., Goodrich, J.K., Gordon, J.I., Huttley, G.A., Kelley, S.T., Knights, D., Koenig, J.E., Ley, R.E., Lozupone, C.A., McDonald, D., Muegge, B.D., Pirrung, M., Reeder, J., Sevinsky, J.R., Tumbaugh, P.J., Walters, W.A., Widmann, J., Yatsunenko, T., Zaneveld, J., Knight, R., 2010. QIIME allows analysis of highthroughput community sequencing data. Nat. Methods 7, 335–336. Christiansen, P., Nielsen, E.W., Vogensen, F.K., Brogren, C.H., Ardo, Y., 2006. Heat resistance of Lactobacillus paracasei isolated from semi-hard cheese made of pasteurised milk. Int. Dairy J. 16, 1196–1204. Christiansson, A., Naidu, A.S., Nilsson, I., Wadstrom, T., Pettersson, H.E., 1989. Toxin production by Bacillus cereus dairy isolates in milk at low-temperatures. Appl. Environ. Microbiol. 55, 2595–2600. De Angelis, M., Di Cagno, R., Huet, C., Crecchio, C., Fox, P.F., Gobbetti, M., 2004. Heat shock response in Lactobacillus plantarum. Appl. Environ. Microbiol. 70, 1336–1346. De Jonghe, V., Coorevits, A., De Block, J., Van Coillie, E., Grijspeerdt, K., Herman, L., De Vos, P., Heyndrickx, M., 2010. Toxinogenic and spoilage potential of aerobic sporeformers isolated from raw milk. Int. J. Food Microbiol. 136, 318–325. De Jonghe, V., Coorevits, A., Van Hoorde, K., Messens, W., Van Landschoot, A., De Vos, P., Heyndrickx, M., 2011. Influence of storage conditions on the growth of Pseudomonas species in refrigerated raw milk. Appl. Environ. Microbiol. 77, 460–470. Delgado, S., Rachid, C.T.C.C., Fernandez, E., Rychlik, T., Alegria, A., Peixoto, R.S., Mayo, B., 2013. Diversity of thermophilic bacteria in raw, pasteurized and selectively-cultured milk, as assessed by culturing, PCR-DGGE and pyrosequencing. Food Microbiol. 36, 103–111. Doughari, H.J., Ndakidemi, P.A., Human, I.S., Benade, S., 2011. The ecology, biology and pathogenesis of Acinetobacter spp.: an overview. Microbes Environ. 26, 101–112. Edgar, R.C., 2010. Search and clustering orders of magnitude faster than BLAST. Bioinformatics 26, 2460–2461. Edgar, R.C., 2013. UPARSE: highly accurate OTU sequences from microbial amplicon reads. Nat. Methods 10 (996-+). Eneroth, A., Christiansson, A., Brendehaug, J., Molin, G., 1998. Critical contamination sites in the production line of pasteurised milk, with reference to the psychrotrophic spoilage flora. Int. Dairy J. 8, 829–834. Eneroth, A., Svensson, B., Molin, G., Christiansson, A., 2001. Contamination of pasteurized milk by Bacillus cereus in the filling machine. J. Dairy Res. 68, 189–196. Ercolini, D., 2013. High-throughput sequencing and metagenomics: moving forward in the culture-independent analysis of food microbial ecology. Appl. Environ. Microbiol. 79, 3148–3155. Fittipaldi, M., Nocker, A., Codony, F., 2012. Progress in understanding preferential detection of live cells using viability dyes in combination with DNA amplification. J. Microbiol. Methods 91, 276–289. Flint, S.H., Bremer, P.J., Brooks, J.D., 1997. Biofilms in dairy manufacturing plant - description, current concerns and methods of control. Biofouling 11, 81–97. Francis, K.P., Mayr, R., von Stetten, F., Stewart, G.S.A.B., Scherer, S., 1998. Discrimination of psychrotrophic and mesophilic strains of the Bacillus cereus group by PCR targeting of major cold shock protein genes. Appl. Environ. Microbiol. 64, 3525–3529. Fricker, M., Skanseng, B., Rudi, K., Stessl, B., Ehling-Schulz, M., 2011. Shift from farm to dairy tank milk microbiota revealed by a polyphasic approach is independent from geographical origin. Int. J. Food Microbiol. 145, S24–S30. Granum, P.E., Lindbäck, T., 2013. Bacillus cereus. In: Doyle, M.P., Buchanan, R.L. (Eds.), Food Microbiology: Fundamentals and Frontiers, 4th ed. ASM Press, Washington, D.C., pp. 491–502. Gunasekera, T.S., Sorensen, A., Attfield, P.V., Sorensen, S.J., Veal, D.A., 2002. Inducible gene expression by nonculturable bacteria in milk after pasteurization. Appl. Environ. Microbiol. 68, 1988–1993. Hayes, M.C., Ralyea, R.D., Murphy, S.C., Carey, N.R., Scarlett, J.M., Boor, K.J., 2001. Identification and characterization of elevated microbial counts in bulk tank raw milk. J. Dairy Sci. 84, 292–298. Huck, J.R., Sonnen, M., Boor, K.J., 2008. Tracking heat-resistant, cold-thriving fluid milk spoilage bacteria from farm to packaged product. J. Dairy Sci. 91, 1218–1228. James, S.J., Evans, J., James, C., 2008. A review of the performance of domestic refrigerators. J. Food Eng. 87, 2–10. Jay, J.M., Loessner, M.J., Golden, D.A., 2005. Modern Food Microbiology. Springer Science+Business Media, Inc. Kable, M.E., Srisengfa, Y., Laird, M., Zaragoza, J., McLeod, J., Heidenreich, J., Marco, M.L., 2016. The core and seasonal microbiota of raw bovine milk in tanker trucks and

29

International Journal of Food Microbiology 266 (2018) 21–30

D. Porcellato et al.

Biodiversity of culturable psychrotrophic microbiota in raw milk attributable to refrigeration conditions, seasonality and their spoilage potential. Int. Dairy J. 57, 80–90. Wang, L.J., Li, P., Zhang, Z.H., Chen, Q., Aguilar, Z.P., Xu, H.Y., Yang, L., Xu, F., Lai, W.H., Xiong, Y.H., Wei, H., 2014. Rapid and accurate detection of viable Escherichia coli O157:H7 in milk using a combined IMS, sodium deoxycholate, PMA and real-time quantitative PCR process. Food Control 36, 119–125.

Vannetten, P., Vandemoosdijk, A., Vanhoensel, P., Mossel, D.A.A., Perales, I., 1990. Psychrotrophic strains of Bacillus cereus producing enterotoxin. J. Appl. Bacteriol. 69, 73–79. Vaz-Moreira, I., Novo, A., Hantsis-Zacharov, E., Lopes, A.R., Gomila, M., Nunes, O.C., Manaia, C.M., Halpern, M., 2011. Acinetobacter rudis sp nov., isolated from raw milk and raw wastewater. Int. J. Syst. Evol. Microbiol. 61, 2837–2843. Vithanage, N.R., Dissanayake, M., Bolge, G., Palombo, E.A., Yeager, T.R., Datta, N., 2016.

30