Risk assessment of use of cracked eggs in Canada

International

Journal

of

Food Microbiology 30 (1996) 1255143

Risk assessment of use of cracked eggs in Canada Ewen C.D. Todd* Bureau of Microbial Hazards, Health Protection Branch, Sir Frederick G. Banting Research Centre, Health Canada. Building Locator: 2204A2, Tunneys Pasture, Ottawa, Ontario, Canada KIA 0L2

Abstract In 1992, Agriculture and Agri-Food Canada (AAFC) amended its Egg Regulations to restrict movement of Canada C eggs (cracks) to federally registered processed egg stations for pasteurization. This was questioned by egg producers and some provinces on economic grounds. It was also in conflict with long-standing practices of marketing eggs in some provinces to retail stores, bakeries, restaurants and institutions or at the farm gate. In order to determine how much of a risk these eggs were to human health, AAFC requested that the Health Protection Branch (HPB) of Health Canada (HC) conduct a risk assessment. On the basis of outbreak data, the main hazard in these eggs was identified as Salmonella. Salmonellae may occasionally be present on shell eggs even after washing, and any Salmonella reaching the membranes can be transferred to an egg mixture through breaking, and will rapidly grow under improper storage conditions. A Relative Risk analysis showed that cracked eggs are 3 to 93 times more likely than untracked shell eggs to cause outbreaks. A probability of illness of 1 in 3800 was derived from the 40 million cracked eggs produced in Canada and not pasteurized and the probable 10 500 illnesses arising from these. This was for the general population, but this would be greater for those who consume many shell eggs or would do so in an unsafe manner, or are more likely to be infected (5% of consumers who eat raw or lightly cooked eggs daily, rural communities with more opportunities for obtaining cracked eggs. and those who are immunocompromised and in institutions). Even though it is not possible to precisely determine the risk of salmonellosis through cracked eggs, this assessment indicated that there was enough of a concern that a management strategy was needed. Eight options for managing the risk were considered and ranked for acceptability by both HC and AAFC. Ideally, all cracked eggs should be broken and pasteurized, but this is impractical in certain regions of the country, and other options, such as sales to food processors operating under Good Manufacturing Practices (GMP), and at

* Tel.:

+ 1 613 9570887;

016%1605/96/$15.00

fax:

+ 1 613 9410280

0 1996 Elsevier

PIZ SO1 68-1605(96)00995-6

Science

B.V. All rights

reserved

126

E.C.D.

Todd / ht.

J. Food Microbiology

30 (1996) 125

143

the farm gate in marked cartons and under controlled conditions, were considered to be acceptable, whereas sales to institutions and bakeries were not. This is the first formal food-related microbiological risk assessment that HC has completed. Although this is a Canadian problem, any country producing eggs has to recognise that despite any regulations controlling the use of cracked eggs, economics will dictate that some of these will be consumed as whole eggs or egg products, and a management plan is desirable to limit hazardous practices associated with these eggs. Keywords: Cracked eggs; Salmonellosis;

Risk assessment; Management;

Canada

1. Introduction Cracked eggs are considered hazardous because their contents are exposed to bacterial pathogens, and they may be used in products that are not thoroughly cooked. For instance, a survey of 45 food preparers in the United States showed that 19% would use cracked eggs for making egg nog (Woodburn and VanDeRiet, 1985). The sale of these eggs in Canada for interprovincial and international trade is allowed only for processed eggs under the Egg Regulations of the Canada Agricultural Products Act (1988). Some provinces, however, have created a provincial grade ‘crack’ for intraprovincial sales, and a few provinces allow internal sales of cracked eggs to institutions, restaurants, retail stores and at the farm gate. On January 1, 1992, AAFC amended the Egg Regulations (Anon, 1992) to restrict the movement of all Canada C eggs (including cracks, see definition below) to federally registered processed egg stations including intraprovincial movement. This prevented the outlet of Canada C eggs for any other sale, meaning that some producers would have to transport these by road and/or ferry up to 2000 km to the nearest breaker. Under these conditions, this would be uneconomical and would encourage local illegal trade. Enforcement was delayed until discussions between provinces and the federal government took place. Each province and territory has a different egg production volume and access to breaking plants, which means for economic reasons some of these wish to allow marketing of cracked eggs at the farm gate or some commercial establishments. Because of this, AAFC requested in 1992 that the Health Protection Branch (HPB) of Health Canada (HC) prepare an assessment of the health risk posed by sales of cracked eggs.

2. The scope of the assessment

and definitions

The risk assessment was conducted by the HPB with limited information based on the international scientific literature and survey data gathered within the provinces and by AAFC. This assessment is to assist risk managers in a decision process for management of cracked eggs. According to the FAO/WHO Expert

E.C.D. Todd 1 ht. .I. Food Microbiology 30 (1996) 125-143

127

Consultation (Anon, 1995b), risk assessments consist of four steps: hazard identification, hazard characterization, exposure assessment, and risk characterization. In this assessment, apart from hazard identification, only components of these steps were available for performing these steps, i.e. infectious dose studies (hazard characterization), likelihood of contamination and growth in eggs, and estimated numbers of cracked eggs produced in Canada and their associated illnesses (exposure assessment), and estimation of risk of salmonellosis from cracked eggs (risk characterization). These are described along with a list of eight options for risk managers for the control of cracked eggs within the provinces. In Canada, an egg may be graded as Canada A if it shows on candling acceptable quality, has a shell free from dirt or stain (except small stain spots), and is untracked (Anon, 1992). Canada B eggs have slightly lower standards than Canada A eggs, but are not cracked. An egg is graded as Canada C if it meets certain standards including being free of dirt, having internal contents that do not leak, and being conveyed to a federally registered egg station; most eggs in this grade are cracked. Provinces may have their own grade cracks, with similar but not necessarily identical definitions, e.g. one provincial regulation states that, (1) the shell is cracked but the internal contents are not leaking; and (2) upon candling, the egg meets the standards for at least Canada C.

3. Risk assessment of cracked eggs 3.1. Hazard ident$cation Seven microbial agents associated with poultry, egg laying barns, eggs and wash water were evaluated as potential hazards in or on cracked eggs. For the first six, no illnesses have been traced to cracked eggs. The seventh, Salmonella, has been the cause of outbreaks associated with this product. 3.1.1. Bacillus cereus B. cereus has been found in egg melange made from cracked or broken eggs, raw and pasteurized liquid whole egg, and bakery products made with liquid whole egg in the United Kingdom (Wood and Waites, 1988). The most likely source of the organism in the melange was contaminated egg shells and cracked eggs. 3.1.2. Campylobacter

The organism is frequently carried by poultry (Jacobs-Reitsma et al., 1995), and can reach eggs. If eggs are dipped in a culture containing hen feces with lo7 C. jejunijml added, and stored at 37”-42°C for 10 min. and then at 4°C for 24-48 h, the organisms will reach both the inner and outer membranes and even the contents of some of these eggs (Doyle, 1984; Chaudhary et al., 1989). As the eggs cool at 4°C the organisms are drawn through the shell. Cracked eggs are easily penetrated by C. jejuni (Chaudhary et al., 1989).

128

E.C.D.

Todd / ht. J. Food Microbiology 30 (1996) 125-143

3.1.3. E. cob 0157:H7 and other verotoxigenic E. coli (VTEC) Although colonization of chick intestinal tracts is possible experimentally with E. coli 0157:H7 (Beery et al., 1985), surveys of pasteurized egg fluid and chicken carcasses in Ontario, Canada, have not revealed the presence of any VTEC (Clarke et al., 1992). 3.1.4. Listeria spp Listeria innocua has been found in egg wash water in Canada (Laird et al., 1991; Farber et al., 1992) and L. monocytogenes can survive in normal wash water if it is artificially introduced (Laird et al., 1991). Egg contents are less likely to be contaminated; L. monocytogenes was present in 2 of 42 samples of commercially broken raw liquid egg in the United States (Leasor and Foegeding, 1989), but none of 50 composite whole egg samples in Canada (Farber et al., 1992). Once L. monocytogenes reaches the yolk, however, as in a cracked or broken egg, it will grow even at 5°C over several days, e.g. 10’ to 108/g in 22 days (Sionkowski and Shelef, 1990). 3.1.5. Staphylococcus aureus In a Japanese study by Suzuki et al. (1981), cracked, broken or dirty eggs stored for a long time without washing or disinfecting were contaminated both with Salmonella and S. aureus. Pasteurization of the melange reduced, but did not eliminate, the organisms. The origin of the S. aureus was not determined but presumably came from the farm. Minor and Marth (1976) report that S. aureus can originate both from poultry and human sources, but they will not grow in eggs stored at I 7.2”C for 12 h or less. S. aureus has caused intoxications in Canada from boiled eggs but these were hard boiled, decorated and stored at room temperature for more than 1 day (Todd, 1991). A similar problem occurred in Sweden when 46 cases (4 fatalities) resulted from consumption of improperly stored hard boiled eggs (Aronsson et al., 1991). 3.1.6. Yersinia enterocolitica Experiments have shown that Yersinia enterocolitica can penetrate the shell and membranes over a period of time similar to Salmonella. The presence of iron (20 ppm) increased the percentage of eggs spoiled (Amin and Draughon, 1990). Storage of eggs at 10°C however, will not prevent the growth of organisms (counts reached 106-‘/g in 21-28 days at 10°C). Even after 7 days at 10°C with no iron supplementation 14% of eggs were contaminated. Y. enterocolitica also grows in egg wash water (Southam et al., 1987). If small traces of iron are present in wash water, the risks of Y. enterocolitica contamination of egg contents is greater and no signs of infection can be noticed in these eggs (Amin and Draughon, 1990). The contents of cracked eggs would be at even greater risk of contamination. 3.1.7. Salmonella Salmonella is present in the environment and on birds in egg laying barns, and surfaces of eggs can be contaminated from feces or litter. In studies of three United

E.C.D.

Todd / ht. J. Food Microbiology 30 (1996) 125-143

129

States laying operations, salmonellae were isolated from 30-72% of environmental samples, e.g. water, ventilation fan, egg belt and egg collectors (Jones et al., 1995). In an examination of 300 Canadian layer flocks, Poppe et al. (1991) found Salmonella in environmental samples associated with 53% of the flocks. In addition, 7 of 90 eggshells (7.8%) before washing contained Salmonella (Jones et al., 1995). This is within the range of 6.3-9.5% contamination of eggshells found when three serotypes of Salmonella were administered orally to laying hens (Cox et al., 1973). 3.1.8. Outbreaks from cracked eggs The following episodes show how cracked eggs used in foods, mainly desserts including puddings and pies, cakes and ice cream, led to salmonellosis: (1) In a large U.S. interstate S. derby outbreak in 1963 raw or undercooked eggs were considered the source (Sanders et al., 1963). One of the two farms supplying the eggs to all the 15 hospitals affected had 4 of 42 slurries of cracked eggs contaminated with the same organism. One sample of poultry feed also contained the same serovar. No salmonellae could be isolated from slurries of Grade A or B eggs from this or other farms. (2) In another hospital outbreak in California a year later, 121 patients had eaten tapioca pudding made with raw egg whites and developed S. heidelberg infection (Ager et al., 1967). Up to 5% of eggs received were ungraded with cracks. The same source of cracked eggs was used to make chocolate eclairs and banana cream pies, both of which had frozen whole eggs as ingredients; some of the 88 cases of S. heidelberg salmonellosis in the Salt Lake City area in a 5-month period in 1964 ate these foods (Ager et al., 1967). The same organism was isolated from 9 of 166 core samples of frozen eggs. (3) In the Basque country of Spain, 15 salmonellosis outbreaks associated with eggs occurred between 1985 and 1987. One of these outbreaks implicated cracked eggs (Perales and Audicana, 1989). S. typhimurium phage type 96 was recovered from one of three cracked eggs obtained as outbreak samples. (4) In Burnaby, British Columbia in 1988 (Khakhria et al., 1991) and Thunder Bay, Ontario in 1991 (Anon, 1991b), illegally obtained cracked eggs were used without heat treatment to make cream fillings for cakes. Over 90 cases of S. enteritidis salmonellosis were attributed to the first outbreak and 53 to the second. (5) In Victoria, Australia, cases of S. typhimurium infection had eaten ice cream in a cafe (Carnie et al., 1991); the ice cream, found contaminated with the same phage type, was prepared from cracked eggs supplied by the cafe owner’s brother who operated a poultry farm. The employee who normally prepared the ice cream was away and the owner made these batches himself. In addition, during the period of the outbreak the freezer had broken down on several occasions allowing the product to thaw. 3.1.9. Egg nog related illnesses There have been many reports of illness arising from egg nog served in hospitals or other institutions. In the past cracked eggs were often involved but not always

130

E.C.D. Todd / ht. f. Food Microbialogy 30 (1996) 125-143

proven to be the source, as,can be seen from the following examples. One hundred and four cases of S. typhimurium infection occurred in Massachusetts where eggs were obtained from the institution’s own flock, but it is not known whether these were cracked or not (Philbrook et al., 1960); in 1979, S. infantis caused an infection in 61 persons in a nursing home in the Kitchener area, Ontario, after consuming egg nog made from cracked eggs. The same serovar was isolated from leftover egg nog. It was not known whether the eggs were contaminated on arrival at the home or the cook introduced the organism during the egg nog preparation as he was also found to be infected (Bollegraaf and Lior, 1980); in three separate outbreaks in Ontario between 1978 and ,1982 egg nog made from raw eggs was contaminated with Salmonella; cracked eggs were specified in one of them. 3.1.10. Canadian outbreaks There were 13 incidents of salmonellosis in Canada involving shell eggs from 1973 to 1991; in at least five of these outbeaks contaminated cracked eggs were used (Table 1). 3.1.11. Conclusion on hazard identification In summary, B. cereus, Campylobacter, Salmonella, S. aureus, and Y. enterocolitica have been found in the environment or on the birds, and could contaminate shells. E. coli 0157:H7/VTEC and L. monocytogenes do not appear to be common in poultry laying barns, although few studies have been done to look for them. However, if any of these pathogens are deposited on shells of cracked eggs, they may be able to reach the membranes. Nevertheless, Salmonella is the only organism that has been conclusively linked to human illness from cracked eggs both in Canada and other countries, and this is the only hazard of concern considered further in this assessment. 3.2. Infectious dose studies The minimum number of Salmonella in eggs required to cause illness is unknown, but seems to be high from the following information: (1) Volunteer studies found that 105-lo8 cells in egg suspensions were required to cause gastroenteritis (McCullough and Eisele, 1951a,b). This is old data, however, that has been criticised for its methodology. Outbreak data for other foods shows between 10’ and 10” (median 102) cells causing illness (Blaser and Newman, 1982); (2) A whole family was ill and one healthy teen died after ingesting 108-lo9 cells of S. typhimurium present in home-made ice cream made from shell eggs from hens in their backyard (Taylor et al., 1984); (3) An outbreak occurred in a German home for the aged with 87 cases of whom 10 elderly persons died. These had eaten a pudding containing whipped egg white with lo7 Salmonella enteritidislml (Anon, 1991d). The first cases occurred 6-8 h after consumption confirming that the dose was large. No Salmonella was isolated from eggs, hens or environmental samples at the farm supplying the eggs;

Salmonella

1973 1977

1978

1979

1982

1985

1986

1986

S. typhimurium

S. infantis

S. infantis, S. schwarzengrund

S. java

Salmonella sp.

Salmonella sp.

3

4

5

6

7

8

Onset

S. enteritidis S. bareilly

serovar

1 2

No.

Ontario

Ontario

Quebec

Ontario

Ontario

Ontario

Ontario Ontario

Province

Home for the aged

1

Farm

Nursing home

Nursing home

Nursing home

Sanitorium food service Manufacturing plant

Where food mishandled

Eggs

Quail eggs

Cracked fresh eggs

Egg nog protein sup plement

Egg nog

Egg nog

Eggs Cakes

Vehicle

Table 1 Salmonellosis from uncooked intact or cracked shell eggs in Canada, 1973-1991

11(?)

1cv

l(7)

46(265)

61(?)

6(185)

lO(69) 44(T)

No. Ill (at risk)

?

?

?

?

Several weeks Average 14 working days

Duration (h)

Eggs probably cracked Contaminated cracked eggs used to make the cakes. Cost of outbreak $23,523 or $535/person Egg nog made with shell eggs Cracked eggs used to prepare egg nog Four patients died with salmonellosis a contributing factor; staff as well as residents ill. A few secondary cases occurred. Shell eggs were used to make the egg nog Six-month old infant ill and hospitalized. The six other asymptomatic farm residents were also positive for S. jaua. Cracked eggs were regularly consumed at the uncle’s farm and the untracked ones were sold. Eggs tested positive for Salmonella (serovar unknown) Uncooked eggs were used as health food ingredients

Comments

53(?)

Reporting of outbreaks from 1988 to 1991 may be incomplete, and none has been documented since 1991.

Cakes

Bakery

Ontario

1991

S. enteritidis

Restaurant

Ontario

1991

13

12

Restaurant

Ontario

1990

S. enteritidis phage type 8 S. enteritidis

11

Tapioca. 60 lemon and chocolate pudding Hollandaise 20(?) sauce Eggs benedict, 9(?) hollandaise sauce

Home for the aged

Quebec

1990

S. enteritidis phage type 22

10

> 90(?)

Cakes

Bakery

British Columbia

1988

S. enteritidis

9

No. 111(at risk)

Vehicle

Where food mishandled

Province

Onset

Salmonella serovar

No.

Table 1 (continued)

Salmonella isolated from patients, staff, cakes and cream fillings. Cracked eggs were used Uncooked shell eggs used as an ingredient

Shell eggs used and sauce insufficiently cooked Two restaurants implicated. One case hospitalized overnight. Insufficiently cooked shell eggs used Cracked eggs used in cream fillings

?

?

?

224

1

Comments

Duration (h)

E.C.D.

Todd / ht. .I. Food Microbiology 30 (1996) 125-143

133

(4) In the United Kingdom, 600000 persons were suspected to have eaten eggs, each infected with between 1 and 100 S. enteritidis without causing salmonellosis (Duguid and North, 1991); (5) In a study of 29 egg-associated salmonellosis outbreaks in the United States, food was improperly refrigerated or inadequately cooked in each one of these, probably allowing growth of Salmonella to large numbers (Anon, 1991~). Therefore, relatively large numbers of Salmonella have probably been present in some egg-containing foods linked to illness. Cakes, tapioca and other puddings, pasta, egg-based sauces, ice cream, Caesar salad and egg nog have been handled and/or exposed to unacceptable time/temperature combinations that allowed Salmonella to grow to levels sufficient to cause outbreaks. In contrast, in Canada, no salmonellosis has been associated with boiled, scrambled or poached eggs. Any salmonellae present would have probably been in low numbers. Otherwise the typical light cooking of these items would not have destroyed the organisms. However, foods with no opportunity for bacterial multiplication after they have been prepared, such as icing and meringues, have been implicated in outbreaks, indicating that growth may not always be necessary for illness to occur. 3.3. Likelihood of contamination 3.3.1. Penetration of eggs by surface Salmonella

Experiments with eggs being placed into bacterial cultures show that the organisms quickly reach the membranes, but will generally not contaminate the yolk for many days. The inner and outer membranes, which are composed of keratin and mucin fibres with albumen in the interstices, prevent most of the bacteria from reaching the contents, but it is the albumen itself that inhibits their multiplication through its anti-bacterial properties. S. enteritidis grows fairly rapidly in the yolk from 1 to 10’ CFU/g in 4 days at 15.5”C (Bradshaw et al., 1990). However, when eggs at 35°C were dipped in a cold suspension of three Salmonella spp. and sampled after drying and storage at 29°C it took most eggs up to 3-4 weeks for the salmonellae to multiply to lo’-10’ CFU/ml of the contents (Stokes et al., 1956; Duguid and North, 1991). According to Wedral et al. (1971), S. typhimurium penetrated almost 100% of eggs in 36 h stored at 25°C but the permeability of the membranes was not changed until after 8 days. Therefore, if intact eggs can be penetrated by salmonellae, then it is likely that cracked eggs allow the bacteria to migrate faster onto the membranes and probably in larger numbers. It would not be necessary, however, for Salmonella to penetrate the yolk to be a hazard. Any organisms lodged under the shell or on the membranes can reach a mixture of broken shell eggs and grow if the storage conditions are inadequate. Therefore, the likelihood for Salmonella to contaminate egg mixtures is high. 3.3.2. Percentage of eggs that are cracked and the effect of washing The level of cracked eggs at the farm level in Queensland, Australia, was 0.3 to 1.4% in the summer and 0.6 to 0.9% in the winter (Davis et al., 1986). Slightly higher levels were reported by Thompson and Hamilton (1986) at an Ontario

134

E.C.D. Todd / ht. J. Food Microbiology 30 (1996) 125-143

grading station with between 1.3 and 6.3% (mean, 3.1%) of 760000 eggs in 35 shipments. About 6.5% of eggs at retail stores in Helsinki, Finland, were cracked or leaky, but none had Salmonella on their shells or in their contents (Hirvela, 1982). However, in a study of New York State eggs by Vadehra et al. (1969), unwashed cracked eggs exposed to S. typhimurium cultures and incubated at 23°C were more prone to infection (3%) than untracked eggs (0%); interestingly, washing eggs of both categories increased the percentage spoiled (cracked, 10%; untracked, 3%), indicating shells might be weakened. The degree of penetration is dependent on the temperature difference between the egg and the washwater, the concentration of microorganisms in the water, the washwater with appreciable soluble iron, the duration of immersion of eggs in the water, the thickness of the eggshell, and the treatment of the egg prior to washing (Panda, 1973; Moats, 1978). Rapid cooling of eggs increases the number of microscopic cracks on shells (Fajardo et al., 1995). Shell weight, percentage shell, shell thickness, specific gravity, and shell weight per unit surface area were higher (P < 0.05) for intact than for cracked and broken eggs (Thompson et al., 1985). When the incubation temperature was reduced to 10°C spoilage decreased (0% for cracked unwashed, untracked washed and unwashed, and 4% for cracked washed) (D’Aoust et al., 1980). Washing removes the cuticle, the first barrier to bacterial penetration (Fajardo et al., 1995) but cleaning agents reduce the bacteria in the washwater. Vadehra et al. (1969) found that quaternary ammonium compounds were more effective than chlorine in reducing spoilage. Detergents at > pH 11 rapidly killed off Salmonella and other bacteria (Kinner and Moats, 1981; Catalan0 and Knabel, 1994). With detergents, the contamination rate could be much reduced, e.g. to 1.1% of eggshells after eggs were washed in recirculated water containing 0.3% detergent (Jones et al., 1995). The wash water was replaced every 4 h. Washing is allowed in Canada and the United States, but not in the United Kingdom (Advisory Committee on the Microbiological Safety of Food, 1993). 3.3.3. Salmonella in intact and cracked eggs Salmonella was found in 0 to 2 percent of washed, commercially processed egg shells in surveys during the 1960s to the 1980s in three U.S. states, British Columbia, Finland, India and Saudi Arabia (Hirvela, 1982; Ching-Lee et al., 1991). In Hawaii in 1989, however, Ching-Lee et al. (1991) found that 9.4% of eggs were contaminated. They claimed this high level was due to better isolation methodology. Nevertheless, none of the contents was positive. In Ontario, D’Aoust et al. (1980) showed that 13% of cracked eggs and 2% of intact shell eggs (both blended shell and contents), and 8% of the contents of cracked eggs contained Salmonella. In a study of batches of five pooled whole and cracked eggs not involved in foodbome disease, Perales and Audicana (1989) in the Basque Country of Spain found that l/20 (5%) cracked eggs (shell with contents) and l/156 (0.6%) whole eggs from farms (shell and contents) were positive for S. enteritidis. In this limited study, cracked eggs appear to present a greater risk to consumers, as the percentage of contaminated cracks was higher than contaminated whole eggs. In another study these authors found S. enteritidis in 4 samples of 24 pools (shells only) at the retail

E.C.D.

Todd / ht. J. Food Microbiology 30 (1996) 125-143

135

level (16.7%); the eggs in these samples were obtained from one store. They considered that the most likely source of the positive shells was uncleaned eggs. 3.4. Likelihood of growth Growth of Salmonella in whole shell eggs is slow and depends on the gradual change in the albumen to allow multiplication there and in the yolk. Even with cracked eggs, as opposed to leakers, growth would not be any faster, as the internal barriers would be the same. However, it is known that when eggs are broken and the yolk and white mixed, as in liquid egg preparations, the antibacterial effect of albumen is neutralized, and growth of pathogens and spoilage organisms can proceed rapidly at suitable temperatures. Therefore, although cracked eggs might harbour more organisms than intact eggs, any type of shell or equipment contamination can introduce Salmonella into liquid egg. Salmonella and other pathogens will grow to 1O6-9/g in egg melange provided there is a long enough storage period at temperatures 2 10°C. If the melange or other egg products are sufficiently heated, the pathogens will be destroyed provided the bacterial load is not too high. However, cooked products have been implicated in salmonellosis. The egg component has either been insufficiently heated, e.g. in meringues, omelettes, sauces, casseroles, lasagna, or it has been added to the final cooked product without any pretreatment, e.g. puddings, ice cream, icing, and filling for cakes. Partially cooked eggs can also be a hazard. In Ethiopia, 79 of 344 students eating unshelled eggs for a communal breakfast were infected with S. newport (Aseffa et al., 1994). The eggs had been boiled in hundreds together and then kept at room temperature for over 14 h before being peeled and consumed. Many of these eggs appeared to be undercooked. It is uncertain what happened in this outbreak, but some form of abuse occurred to allow the Salmonella to reach infective dose levels in individual eggs overnight. Unfortunately, no eggs remained for testing. 3.5. Estimation of cracked eggs in Canada and their associated cases 3.5.1. Estimation of volume of cracked eggs In 1985, a survey at an Ontario grading station showed that 3.1% were cracked (Thompson and Hamilton, 1986). However, there may have been some improvement since then based on the 1994 AAFC Poultry Market Review (Anon, 1995a). Of the 5016 million eggs that were graded across Canada in 1994, 4636.8 million were Canada A (92.4%), 39.6 million were Canada B (0.8’!0), 123.6 million were Canada C (2.S”/o, and 216.0 million were nest run eggs for processing (4.3%) (Anon, 1995a). However, there was considerable variation in the percentage of Canada C eggs graded by province and territory (up to 3.3%). Most, but not all, of Canada C eggs are cracked; so, the 2.5% would represent a maximum figure. In addition, Canada A eggs may develop cracks during subse-

136

E.C.D. Todd / ht. J. Food Microbiology 30 (1996) 125-143

quent handling, but the risk of contamination by Salmonella is much less because there is no exposure to fecal matter and they are packed in cartons. Therefore, these post-grading cracked eggs are not considered in the assessment. This 2.5%, however, only applies to graded eggs; if this is typical for all eggs produced annually in Canada (5736 million), the total number of cracked eggs would be 143 million up to the grading stage. 3.5.2. Estimation of number of salmonellosis cases associated with cracked eggs From Table 1 there is reasonable evidence that in at least 5 of the 13 outbreaks in Canada involving shell eggs, contaminated cracked eggs were used. Even though this does not seem to be a large number over the 18 years from 1973 to 1991, it should be recognized that typically only a small percentage of eggs is cracked. If the proportion of eggs not sent to the breakers in 1990 is used for comparative analysis, the percentage is 2.7 (3960 million Canada A and B compared with 108 million Canada C eggs, Anon, 1991a). From these figures and numbers of outbreaks caused by Salmonella-contaminated cracked (5) and whole shell (8) eggs in Table 1, the increase in risk of outbreaks from the use of cracks is estimated to be 23:l (5 x 97.3/8 x 2.7). Since this estimation of relative risk (RR) is subject to uncertainty in the numbers of outbreaks and cracked eggs, a range of RR values was computed by first constructing a 95% confidence interval for the observed proportion of outbreaks from cracks (5/13), and then combining this result with a range of possible proportions of cracked to untracked eggs (2%-5%). Also, an assumption was made that the rate of reporting outbreaks arising from cracked eggs would be the same as that from intact eggs. This analysis yields the interval [3, 931 of RR estimates. Therefore, even if the lowest impact by cracked eggs is considered, these are three times more likely than untracked shell eggs to cause outbreaks. If there are about 150 cases arising from consumption of cracked eggs occurring in Canada from 1988 to 1991 (based on data in Table l), this represents 30 cases/year. Other outbreaks may have occurred since then but are unreported. This is no doubt a considerable underestimation. This would especially apply to any people at the farm who consume cracked eggs rather than discard them. It has been argued that a more realistic case estimate could be obtained by multiplying reported outbreak cases by 350, as suggested by Todd (1989) based on the median case figure from studies of published estimates and population surveys. If this is done with the 30 cases, then the number of actual cases could be as high as 10 500/year. 3.5.3. Conclusion of the exposure assessment In conclusion, Salmonella can be found on the shells of eggs, and they occasionally penetrate these to reach the membranes. If washing of fecally soiled eggs is incorrectly done, bacteria in the washwater, including Salmonella, can be absorbed into the shell, rather than being removed and destroyed. From the results of several studies, the presence of cracks in eggs increases the likelihood

E.C.D.

Todd / Znt. J. Food Microbiology 30 (1996) 125- 143

137

of contamination inside the shell, but the conditions for growth within the eggs are the same for both cracked and intact eggs. The greatest opportunity for growth is when eggs are broken and the contents mixed. If cracked eggs are broken to be used for egg dishes with minimal or no cooking, there is more chance that the egg melange will be contaminated and rapid growth of Salmonella occur. This is why it is recommended that pasteurized liquid egg be used in these kinds of egg dishes at the commercial level. Each year, about 143 million cracked eggs are produced in Canada, and an estimated 10 500 salmonellae cases arise from their consumption. Although high numbers of salmonellae appear to have been present in eggs or egg products implicated in many outbreaks involving cracked eggs, it is also probable that < 100 cells could have caused infections in other situations, especially where elderly persons were involved. 3.6. Estimation of risk of salmonellosis from cracked eggs 3.6.1. Risk of cracked eggs to the population The number of cracked eggs produced in Canada annually is probably about 143 million. However, 93.6 million Canada C eggs are sent to pasteurizing plants (Anon, 1995a). In addition, a number of ungraded eggs are also pasteurized (370.8 million) and if it is assumed that the proportion of these that are cracked is about 2.5%, a further 9.4 million are pasteurized. This leaves 40 million for shell egg consumption. Since about 10 500 cases of salmonellosis are estimated to occur each year in Canada arising from consumption of Canada C eggs or products made with these eggs, the risk of illness is about 1 in 3800 (10 500/40 million). This assumes that the more cracked eggs are consumed, the higher the risk of illness, even though the occurrence of illness depends on a specific chain of events, such as presence of Salmonella in the eggs, opportunity for growth, and incomplete cooking, which may only happen periodically. The risk for the general population is lower than this because, (1) only some cracked eggs are contaminated with Salmonella; (2) most cracked eggs or foods made from cracked eggs used in the foodservice establishment or home are refrigerated until use; and (3) most egg products are cooked which would destroy at least some of the salmonellae present. In contrast, some persons are at much higher risk: (a) the young, elderly, immunocompromised, and ill persons, (b) those that eat incompletely cooked shell eggs daily (about 5% of the population or 1.35 million, Thorne, 1991). and (c) those in rural areas or farms who have more opportunities for obtaining cracked eggs. When this assessment was initiated, 1990 data was examined and the cracks occurred on average at 2.9% with 50 million cracked eggs not pasteurized (Anon, 1991a). This decreased to 2.5% in 1994. Therefore, some improvements have occurred in the 4-year period. No incidents involving cracked eggs have been recorded since 1991, but this does not mean none occurred. Surveillance data is incomplete at the national level since this time.

138

E.C.D.

Todd / ht. J. Food Microbiology 30 (1996) 125- I43

3.6.2. Conclusions on the risk A Relative Risk analysis likely than untracked shell illness from consumption of have a greater risk.

of illness showed that cracked eggs are 3 to 93 times more eggs to cause outbreaks. On average, the risk of cracked eggs is 1 in 3800, but certain populations

4. Managing the use of cracked eggs 4.1. Eight options and iheir ranking Eight options were proposed by provinces and the egg producers. HC ranked these from 1 to 8, based on the population exposed to the hazard, historical reports of illness, likelihood of abuse and growth of Salmonella in the eggs, and compatiblility with existing guidelines and advice. (1) Cracked eggs will be pasteurized in existing egg breaking stations and new breaking stations will be constructed. (2) Cracked eggs will not be allowed to be sold except to egg breaking stations for pasteurized liquid egg. There are currently 13 egg breaking stations across Canada, and no new egg breaking stations will be built. (3) Cracked eggs may be sold directly to a food processing plant operating under Good Manufacturing Practices (GMPs), if the product in which the cracked eggs are used is guaranteed by the manufacturer to receive a heat treatment equivalent to that specified in the AAFC Processed Egg Regulations. (4) Cracked eggs may be sold from registered egg stations in appropriately stored and marked cartons directly to consumers, if the station is situated on the producer’s premises and is grading eggs produced solely on these premises. (5) Cracked eggs may be sold from registered egg stations in appropriately stored and marked cartons directly to consumers from registered egg stations grading eggs from other producers. (6) Cracked eggs may be sold from registered egg stations to retail stores for resale to consumers; however, no direct sales to institutions or bakeries would be permitted. (7) Sales of cracked eggs from egg stations to restaurants would be permitted by provinces requesting this. (8) Cracked eggs would be sorted by candling into those with fine cracks (hairline) and those with visible cracks, with the former being acceptable for sale to the public at the retail level and the latter being sold exclusively for pasteurized liquid egg. 4.2. Recommendations for risk managers at HC and AAFC Options 1 to 3 are unconditionally acceptable, but Options 4 to 6, only under specified conditions. Options 1 and 2, requiring all cracked eggs to be pasteur-

E.C.D.

Todd 1 ht. J. Food Microbiology 30 (1996) 125-143

139

ized in egg breaking stations, present the least risk and are the best control over the potential hazards from cracked eggs, but the first one requires the building of extra egg breaking stations, which would incur considerable cost. In one suggested HACCP approach for controlling hazards in the production, handling, processing and storage of eggs and egg products, cracked eggs are removed and sent to egg breaking plants (Bryan et al., 1991). This is the policy in the United States and Europe. In fact, most food processing companies in Canada that incorporate eggs as ingredients into products use pasteurized liquid egg or pasteurized white or yolk. They do not even use intact Grade A shell eggs because of the risk of contamination of incoming ingredients or finished product. However, it is recognised that there are economic issues that make pasteurization impractical in some regions of the country. Therefore, the risk of infection should be reduced through the most effective controls that can be operated in those regions. Option 3 will only apply if manufacturers wish to use cracked eggs, and currently most do not, and they can satisfy the regulatory agency that they operate under GMPs. The limited sale of cracked eggs directly to consumers from registered egg stations at the farm gate (seller of own product, Option 4), and seller of multiple producers (Option 5), will better control the existing situation. The cracked eggs would be sold refrigerated, and would be accompanied by appropriate handling and cooking instructions, and would not be offered to commercial operations. These options are consistent with present provincial legislation and would be permitted in all areas of Canada. The sale from all registered egg stations to retailers (Option 6) would only be allowed in areas where it was not economically feasible to ship cracked eggs to breakers. Some provinces are presently permitting the sale of cracked eggs at retail stores. These options would only be permitted in areas where it is recommended by provincial health agencies and where assured controls are given. The eggs would need to be sold in appropriately marked cartons, as at the farm gate. Sales would only be permitted to restaurants if all concerns with respect to control as related to consumer health and safety were adequately addressed (Option 7). Option 8, the introduction of a new grading system, is not feasible at this time, and penetration of Salmonella through micro-cracks in shells has been shown (Fajardo et al., 1995).

5. Conclusions Several lines of action with a view to reducing risks from use of cracked eggs are already starting to be followed. AAFC has prepared an amendment to the Egg Regulations to permit the marketing of cracked eggs from registered egg grading stations and will enter into formal agreements with each of the provinces in respect to the safe marketing of cracked eggs. AAFC is working with the egg industry to promote an understanding of the problem and to cooperate in im-

140

E.C.D.

Todd / ht. J. Food Microbiology 30 (1996) 125-143

proving egg production in Canada through the implementation of HACCP. There is also consultation with provincial health and agriculture authorities and municipalities on how to minimize sales of cracked eggs directly to the public, and how to issue educational information, especially for susceptible groups of the population, on the risks associated with the consumption of cracked eggs. The process to develop options was a lengthy one with many meetings between HC and AAFC, and consultations with the provinces by AAFC. These delays may be inevitable in some situations and therefore preparation of risk assessments and agreement on risk management strategies may be a long-term process. This is the first formal food-associated microbiological risk assessment to be completed by the HPB, and may be one approach for future assessments. However, it would also be desirable to perform modelling experiments to assess the growth of Salmonella in intact eggs and egg mixtures at different temperatures. Whenever relevant new data become available, this risk assessment will be reviewed and updated.

Acknowledgements This document was only possible through contributions from Food Production and Inspection Branch, AAFC, and in particular from Art Allen, Chief, Egg and Egg Products, Food Inspection Directorate and illness data from the Foodborne Disease Reporting Centre, HPB. The statistical analysis for Relative Risk was done by William Ross, Bureau of Biostatistics and Computer Applications, HPB, HC. I also appreciate the critical reviews given by Helene Couture and Joost Harwig, HPB, HC, and Tom Feltmate, Judy Fenwick, Wesly Kowalski of the Food and Inspection Directorate, and Case Poppe and Paul Shadbolt of the Animal and Plant Health Directorate, AAFC.

References Advisory Committee on the Microbiological Safety of Food (1993) Report on Salmonella in eggs. HMSO, London, pp. l-58. Ager, E.A.. Nelson. K.E., Galton, M.M., Boring III, J.R. and Jernigan. J.J. (1967) Two outbreaks of egg-borne salmonellosis and implications for their prevention, J. Am. Med. Assoc. 199, 3722378. Amin, M.K. and Draughon, F.A. (1990) Infection of shell eggs with Yersiniu enterocolitica. J. Food Protect. 53, 8266830. Anon (1991a) Poultry Market Review, 1990. Poultry Div., Commodity and Food Industry Directorate, Agriculture Canada, Ottawa, Ontario, pp. l-67. Anon (1991b) Sulmonellu outbreak investigation widens, Thunder Bay District Health Unit, Thunder Bay, Ontario. Anon. (19916 Salmonella task force says 22 investigations completed. Food Chem. News, Oct. 14, pp. 22-23. Anon (1991d) Lethal salmonellosis in old peoples home through pudding with whipped eggs. Newsletter No. 30. WHO Surveillance Program of Control of Foodborne Infections and Intoxications in Europe, Institute of Veterinary Medicine, Berlin, p. 1.

E.C.D.

Todd / Int. J. Food Microbiology 30 (1996) 125-143

141

Anon. (1992) Egg regulations, amendment. Canada Agricultural Products Act, Canada Gazette, Part II, 126, 19-26. Anon (1995a) Poultry Market Review, 1994. Poultry Div., Agricultural Industry Services Directorate, Agriculture and Agri-Food Canada, Ottawa, Ontario, pp. l-70. Anon. (1995b) Application of risk analysis to food standards issues. Report of the Joint FAOjWHO Expert Consultation, Geneva, Switzerland, March 13-17, 1995. WHO/FNU/FOS/95.3, Geneva. Aronsson, B., Hallander, H., Osterman, B., Kvamfors, P., Bergendahl, K., Rajs. J. and Miillby, R. (1991) Fatal outcome in a nosocomial outbreak of Staphylococcus aureus food-poisoning and foodbome infection. In: J. Jeljaszewicz and P. Ciborowski (editors), The Staphylococci. Zbt. Bakt. Suppl 21. Gustav Fischer Verlag, Stuttgart, pp. 263-265. Aseffa, A., Mengistu, G. and Tiruneh, M. (1994) Salmonella newport: outbreak of food poisoning among college students due to contaminated undercooked eggs. Ethiop. Med. J. 32, l-6. Beery. J.T., Doyle, M.P. and Schoeni, J.L. (1985) Colonization of chicken ceca by Escherichia co/i associated with hemorrhagic colitis. Appl. Environ. Microbial. 49, 310-315. Blaser, M.J. and Newman, L.S. (1982). A review of human salmonellosis. 1. Infective dose. Rev. Infect. Dis. 4, 1096-I 106. Bollegraaf, E. and Lior, H. (1980) Quarterly report on human salmonellosis in Canada: third quarter 1979. Can. Dis. Week. Rep. 6. 53-59. Bradshaw, J.G., Shah, D.B.. Forney, E. and Madden, J.M. (1990) Growth of Salmonella enteritidis in yolk of shell eggs from normal and seropositive hens. J. Food Protect. 53, 1033- 1036. Bryan. F.L.. Bartleson, C.A.. Cook, O.D., Fisher. P., Guzewich, J.J., Humm, B.J., Swanson, R.C. and Todd, E.C.D. (1991) Procedures to Implement the Hazard Analysis Critical Control Point System. Int. Assoc. Milk, Food Environ. Sanitarians, Ames, IA. Catalano, C.R. and Knabel, S.J. (1994) Destruction of Salmonella enteritidis by high pH and rapid chilling during simulated commercial egg processing. J. Food Protect. 57, 5922603. Camie, J., Lester, R., Wilby, R., Forsyth, J., Lightfoot, D. and Tan, A. (1991) Salmonella typhimurium 135 outbreak, Victoria, 1991. Commun. Dis. lntell.. Dept. of Health, Woden, Australia 15, 195- 196. Chaudhary, A., Gupta, L.K. and Sharma. S. (1989) Penetration of Campylobacter jejuni at different storage temperatures in membranes and contents of cracked eggs. MIRCEN J. 5, 1999204. Ching-Lee, M.R., Katz, A.R., Sasaki. D.M. and Minette, H.P. (1991) Salmonella egg survey in Hawaii: evidence for routine bacterial surveillance. Am. J. Public Health. 81. 764766. Clarke. R.C., Read, S.C.. McEwen, S.A., Lynch, J., Schoonderwoerd, M.. Lior, H. and Gyles. C.L. (1992) Isolation of verocytotoxin-producing Escherichia coli from animal and food products, In: E.C.D. Todd and J. M. MacKenzie (editors), Proc. Workshop on Methods to Isolate Escherichia coli 0157:H7 and Other Verotoxigenic Organisms from Foods. Polyscience Publications, Morin Heights, Quebec, pp. 12I- 129. Cox. N.A.. Davis, B.H., Watts, A.B. and Calmer, A.R. (1973) Salmonella in the laying hen. I. Salmonella recovery from viscera, feces, and eggs following oral inoculation. Poult. Sci. 52, 661-666. D’Aoust, J.-Y., Stotland, P. and Randall, C.J. (1980) Salmonella in ‘grade cracks’ shell eggs. Can. Inst. Food Sci. Technol. J. 13, 184-187. Davis, B.M., O’Rourke, P.K. and Stephenson, H. (1986) The quality of eggs on farms and at retail outlets in North Queensland during summer and winter. Qld. J. Agric. Anim. Sci. 43, 59-66. Doyle, M.P. (1984) Association of Campylobacter jejuni with laying hens and eggs. Appl. Environ. Microbial. 47. 533-534. Duguid, J.P. and North, R.A.E. (1991) Eggs and Salmonella food poisoning: an evaluation. J. Med. Microbial. 34, 65-72. Fajardo, T.A., Anantheswaran, R.C., Puri, V.M. and Knabel, S.J. (1995) Penetration of Salmonella enteritidis into eggs subjected to rapid cooling. J. Food Protect. 58, 473-477. Farber, J.M.. Daley, E. and Coates, F. (1992) Presence of Listeria spp. in whole eggs and wash water samples from Ontario and Quebec. Food Res. Int. 25, 143- 145. Hirvela, V. (1982) Hygienic aspects on egg quality. Suomen. Ehiinlllkarilehti 88, 4455455.

142

E.C.D.

Todd / Int. J. Food Microbiology 30 (1996) 125-143

Jacobs-Reitsma, W.F., van de Giessen, A.W., Bolder, N.M. and Mulder, R.W.A.W. (1995) Epidemiology of Cumpylobacter spp. at two Dutch broiler farms. Epidemiol. Infect. 114, 413-421. Jones, F.T., Rives, D.V. and Carey, J.B. (1995) Salmonella contamination in commercial eggs and an egg production facility. Poult. Sci. 74, 753-757. Khakhria, R., Duck, D. and Lior, H. (1991) Distribution of Salmonella enteritidis phage types in Canada. Epidemiol. Infect. 106, 25-32. Kinner, J.A., and Moats, W.A. (1981) Effect of temperature, pH, and detergent on survival of bacteria associated with shell eggs. Poult. Sci. 60, 761-767. Laird, J.M., Bartlett, F.M. and McKellar, R.C. (1991) Survival of Listeria monocytogenes in egg washwater. Int. J. Food Microbial. 12, 115- 122. Leasor, S.B. and Foegeding, P.M. (1989) Lisferiu species in commercially broken raw liquid whole egg. J. Food Protect. 52, 7777780. McCullough, N.B. and Eisele, C.W. (1951a) Experimental human salmonellosis. I. Pathogenicity of strains of Salmonella meleagridis and Salmonella anatum obtained from spray-dried whole egg. J. Infect. Dis. 88, 278-289. McCullough, N.B. and Eisele, C.W. (1951b) Experimental human salmonellosis. III. Pathogenicity of strains of Salmonella newport, Salmonella derby and Salmonella bareilly obtained from spray-dried whole egg. J. Infect. Dis. 89, 209-213. Minor, T.E. and Marth, E.H. (1976) Staphylococci and Their Significance in Food. Elsevier, New York, p. 297. Moats, W.A. 1978. Egg washing: a review. J. Food Protect. 41, 919-925. Panda, P.C. 1973. Microbial contamination of shell eggs. Poultry Advisor 6, l-20. Perales, I., and A. Audicana. 1989. The role of hens’ eggs in outbreaks of salmonellosis in north Spain. Int. J. Food Microbial. 8, 175-180. Philbrook, F.R., MacCready, R.A., Van Roekel, H., Anderson, E.S., Smyser, Jr., C.F., Sanen, F.J. and Groton, W.M. (1960) Salmonellosis spread by a dietary supplement of avian source. N. Engl. J. Med. 263, 7133718. Poppe, C., Irwin, R.J., Forsberg, C.M.. Clarke, R.C. and Oggel, J. (1991) The prevalence of Salmonella enteritidis and other Salmonella spp. among Canadian registered commercial flocks. Epidemiol. Infect. 106, 259-270. Sanders, E., Sweeney, Jr., F.J., Friedman, E.A., Boring, J.R., Randall, EL. and Polk, L.D. (1963) An outbreak of hospital-associated infections due to Sulmonella derby. J. Am. Med. Assoc. 186, 9844986. Sionkowski, P.J. and Shelef, L.A. (1990) Viability of Listeriu monocytogenes strain Brie-l in the avian egg. J. Food Protect. 53, 15-17. Southam, G., Pearson, J. and Holley, R.A. (1987) Survival and growth of Yersiniu enterocolitica in egg wash water. J. Food. Protect. 50, 1033107. Stokes, J.L., Osborne, W.W. and Bayne, H.G. (1956) Penetration and growth of Salmonella in shell eggs. Food Res. 21, 510-518. Suzuki, A., Kawanishi, T., Konuma, H., Takayama, S., Imai, C. and Saitoh, J. (1981) Salmonella and Staphylococcus aureus contamination in liquid whole eggs. Studies on the bacterial contamination of liquid (frozen) whole eggs. IV. J. Food Hyg. Sot. Jpn. 22, 223-232. Taylor, D.N., Bopp, C., Barkness, K. and Cohen, M.C. (1984). An outbreak of salmonellosis associated with a fatality in a healthy child: a large dose and severe illness. Am. J. Epidemiol. 119, 907-912. Thompson, B.K. and Hamilton, R.M.G. (1986) Relationships between laboratory measures of egg shell strength and breakage of eggs collected at a commercial grading station. Poultry Sci. 65, 1877-1885. Thompson, B.K., Hamilton, R.M.G. and Grunder, A.A. (1985) The relationship between laboratory measures of egg shell quality and breakage in commercial egg washing and candling equipment. Poultry Sci. 64, 901-909. Thome, G.M. (1991) Salmonella: the chickens and the eggs. Clin. Microbial. Newslett. 13, 65-68. Todd, E.C.D. (1989) Preliminary estimates of costs of foodbome disease in Canada and costs to reduce salmonellosis. 4. Food Protect. 52, 586-594.

E.C.D.

Todd 1 ht. J. Food Microbiology 30 (1996) 125-143

143

Todd, E.C.D. (1991) Foodborne disease in Canada; a IO-year summary, 197551984. Health Protection Branch, Health and Welfare Canada. Polyscience Publications, Morin Heights, Quebec. Vadehra, D.V., Baker, R.C. and Naylor, H.B. (1969) Salmonella infection of cracked eggs. Poultry Sci. 48, 9544951. Wedral, M.W., Vadehra, D.V. and Baker, R.C. (1971) Mechanism of bacterial penetration through the eggs of Callus gallus. 2. Effect of penetration and growth on permeability of inner shell membrane. J. Food Sci. 36, 520-522. Wood, S.L. and Waites, W.M. (1988) Factors affecting the occurrence of Bacillus cereus in liquid whole egg. Food Microbial. 5, 1033107. Woodburn, M. and VanDeRiet, S. (1985) Safe food: care labeling for perishable foods. Home Econ. Res. J. 14, 1- IO.