Fatal firearm-related injury surveillance in Maryland

Fatal Firearm-Related Injury Surveillance in Maryland Brian Wiersema, Colin Loftin, PhD, Robert C. Mullen, MPH, Erich M. Daub, Monique A. Sheppard, MS, John E. Smialek, MD, David McDowall, PhD Context:

Maryland began a statewide firearm-related injury surveillance system in 1995. The system now focuses on firearm-related deaths; a system to monitor nonfatal injuries is being developed. The system is passive; it accesses, integrates, and analyzes data collected by Maryland’s Office of the Chief Medical Examiner, Maryland State Police, and Division of Health Statistics.

Objective:

To evaluate the surveillance system’s ability to ascertain cases in the absence of a standard for the true number of cases.

Design:

Link records of the same firearm-related death captured by the surveillance system’s multiple data sources, comparing the rate of false positives and false negatives, and assessing errors in linkage variables.

Setting:

Maryland, 1991–1994.

Participants: All deaths occurring in the state of Maryland as a result of a firearm-related injury. Main Outcome Measures:

Sensitivity and positive predictive value.

Results:

The system is extremely sensitive, detecting 99.61% of cases, and it has a very high positive predictive value, with 99.87% of the cases identified from medical examiner’s office data being confirmed as actual cases.

Conclusions: Maryland’s database of information from the medical examiner’s office is highly accurate for ascertaining firearm-related deaths that occur in the state. A unique identifier common across data sources would ease record linkage efforts, and improve the system’s ability to monitor firearm-related deaths. Medical Subject Headings (MeSH): firearms, mortality, population surveillance, data collection, medical record linkage, evaluation studies (Am J Prev Med 1998;15(3S): 46 –56) © 1998 American Journal of Preventive Medicine

Introduction

F

irearms have been the leading cause of injuryrelated deaths in Maryland since 1991.1 It is one of only a few states where the number of firearmrelated injury deaths exceeds the number of motor vehicle injury deaths.2 Mirroring national patterns, firearm-related deaths in Maryland disproportionately affect youths, males, and blacks. Although the overall From the Violence Research Group (Wiersema), Department of Criminology and Criminal Justice, University of Maryland, College Park, Maryland 20742-8235; Violence Research Group (Loftin, McDowall), School of Criminal Justice, University at Albany, State University of New York, Albany, New York 12222-0001; Office of Injury and Disability Prevention (Mullen, Daub, Sheppard), Maryland Department of Health and Mental Hygiene, Baltimore, Maryland 21201-2323; and Office of the Chief Medical Examiner (Smialek), State of Maryland, Baltimore, Maryland 21201-1020. Address Correspondence to: Mr. Wiersema, Violence Research Group, Department of Criminology and Criminal Justice, University of Maryland, College Park, Maryland 20742-8235.

46

Am J Prev Med 1998;15(3S) © 1998 American Journal of Preventive Medicine

age-adjusted rate of deaths from firearm-related injuries has remained stable since 1991 (about 15 per 100,000 residents), the rate of firearm-related deaths has increased nearly 17% among youths aged 15 to 24 years, from 29.5 deaths per 100,000 residents in 1991 to 34.4 per 100,000 in 1996.1 Firearm-related deaths are concentrated in two areas of the state: the city of Baltimore and Prince George’s County, a large county adjacent to Washington, DC. In 1996, 81% of firearmrelated homicides and 63% of firearm-related deaths in the state occurred in these jurisdictions.1 These trends highlighted the need in Maryland for a statewide surveillance system to monitor the magnitude and composition of this important problem. Although mortality data collection efforts by the state medical examiner, state health department, and state police had been routine and well-accepted, the data had never been linked or subjected to case-by-case comparison.

0749-3797/98/$19.00 PII S0749-3797(98)00055-5

The Maryland Department of Health and Mental Hygiene began addressing these needs in 1995, when it created a passive surveillance system that monitors firearm-related deaths by accessing, integrating, and analyzing data collected by the Maryland Office of the Chief Medical Examiner and other state agencies. The system now monitors only fatal injuries, but a system to monitor nonfatal injuries is under development. The system addresses three primary concerns: the problem of firearm-related deaths in Maryland, the heavy concentration of firearm-related deaths in Baltimore and Prince George’s County, and the need for firearmrelated data that could be used to evaluate the effectiveness of legislative initiatives such as the establishment of a handgun roster board, banning of assault weapons, and most recently, passage of a law limiting firearm purchases to one gun per month.

Surveillance Activities The Maryland Firearm-Related Injury Surveillance System is designed to monitor the public health impact of firearm injuries and deaths and to provide a means for evaluating the effects of prevention policies in the future. In Phase I, which began in 1995, the system monitored firearm-related deaths. In Phase II, now under development, the system will begin also to monitor nonfatal firearm-related injuries, using information from hospital discharge data and emergency department sources. This article focuses only on the state’s current activities to monitor firearm-related deaths.

Coverage The surveillance system has two levels of coverage and detail for the collection of information on firearmrelated deaths: ●

●

Level 1. The system ascertains all firearm-related deaths occurring in the state. Level 1 information is brief and basic, but it provides statewide coverage. Level 2. The system focuses on firearm-related deaths in the city of Baltimore and Prince George’s County. The data include information about weapons and circumstances from police records on firearm-related fatalities. Level 2 information is highly detailed, but it covers only two areas within the state.

Data Sources The firearm-related two sources of data: Office of the Chief (2) Supplementary

mortality surveillance system taps (1) data maintained by the state’s Medical Examiner (OCME); and Homicide Reports (SHR), data

The Maryland Handgun Roster Board is a panel whose objective is to publish a list of handguns permissible for sale or manufacture in the state, based on a judgment of the handgun’s usefulness for legitimate sporting, self-protection, and law-enforcement purposes.

compiled by the Maryland State Police. A third source of information, computerized death certificate data from the Maryland Department of Health and Mental Hygiene’s Division of Health Statistics, is also described, because it is used in this report as a basis for comparing all types of firearm-related deaths. Medical examiner data. Maryland operates a statewide medical examiner system to certify deaths by homicide, suicide, and accident. It is also charged with investigating sudden deaths of persons in apparent good health or unattended by a physician, and any other deaths of a suspicious or unusual manner. The system is headed by the OCME. Deputy medical examiners in each county are responsible for the initial investigation and certification of deaths in their counties, and they may determine the cause of death. The OCME’s forensic pathology staff members perform autopsies when necessary to establish the cause of death. Because autopsies are required in all cases of homicide that occur in the state, the OCME staff members complete the records in these cases. In cases of suicide and unintentional death, autopsies are carried out when warranted by the circumstances. This involves discussion between the deputy medical examiners and OCME staff. If no autopsy is conducted, the deputy medical examiner completes the death record and forwards a copy to the OCME within 3 months of the death. The OCME database is thus designed to have a complete record of all homicides, suicides, and deaths due to unintentional and undetermined causes within 3 months of certification of death. The OCME database was selected as the foundation for the surveillance system for two reasons: (1) it records the events in a timely manner; and (2) it is a relational database stored on a minicomputer that permits automated data extractions. Police data (homicide only). Maryland operates a law enforcement– based system for recording homicide data that is roughly parallel to the OCME homicide database. It relies primarily on voluntary submission of completed Uniform Crime Reporting forms from each police jurisdiction to the Maryland State Police. Part of this system includes the SHR, a separate form requesting basic (individual-level) information on each victim of murder and nonvehicular manslaughter. The Maryland State Police’s Central Records Division is responsible for receiving and coding these paper reports. Each month, the surveillance system staff receives photocopies of SHRs from the Uniform Crime Reporting unit. To speed access to the data and to preserve textual comments not keyed into computers, the surveillance system staff performs its own entry of the data on each homicide victim. Thus, the SHR data are not as timely as those from the OCME’s file. The OCME database includes data on homicides within 3 to 4 days

Am J Prev Med 1998;15(3S)

47

of examination; the SHR data are compiled monthly. Maryland has no statewide, police-generated database providing basic SHR-type information on suicide. Death certificate (vital statistics) data. The Maryland Department of Health and Mental Hygiene’s Division of Health Statistics maintains the state’s mortality statistics database, which includes information on deaths due to firearm-related causes. Death certifiers forward copies of death certificates to the Division of Health Statistics where the data are coded and entered into computer files. This office then assigns cause-of-death classifications, according to International Classification of Diseases, 9th Revision (ICD) definitions. Death certificate data are not used for firearm-related surveillance because they become available only after an extended period of time (usually 6 months to a year after death). Because timeliness is an important feature in any surveillance system, the death certificate data are not part of the main surveillance database but rather are used for retrospective evaluations of the system.

Definitions The surveillance system is intended to capture all fatal firearm injuries occurring within Maryland. In addition to detecting firearm-related deaths, the surveillance system also captures nonfirearm-related homicides and suicides to provide a baseline for comparison. A firearm-related death is defined as any fatal injury resulting from the discharge of a weapon from which a projectile is propelled by explosives. Manner of death is the term used to distinguish homicide, suicide, unintentional fatal injury, and fatal injury of undetermined manner. Suicide is any intentionally self-inflicted fatal injury. Firearm-related deaths of undetermined manner are those that are pending classification or those that cannot be classified as homicides, suicides, or unintentional deaths. The definition of homicide varies according to data source. According to the OCME, homicide is death resulting from an act or an omission by another person. Homicide, thus defined, includes legal interventions (law enforcement officials acting in the line of duty), intentional deaths by civilians that may be legally justifiable or excusable, and unintentional killings of persons by other persons. Motor vehicle crash–related deaths caused by other persons are not classified as homicides. In homicide data based on police sources, a distinction is usually made between criminal and noncriminal homicide. Criminal homicide includes murder, non-negligent manslaughter, and manslaughter due to gross negligence. Unintentional firearm-related deaths are those that result from unintentional events or that are otherwise not purposeful.

48

Figure 1. Flow of information in the Maryland FirearmRelated Injury Surveillance System.

Surveillance Methods Because this surveillance system is passive, its methods are simple. At regular intervals, the surveillance system accesses data that meet the case definition criteria, standardizes data formats across sources, links data records across sources when appropriate, and focuses analysis and reporting on firearm-related injury events (Figure 1).

American Journal of Preventive Medicine, Volume 15, Number 3S

Data accession. Every week (or other desired interval), surveillance system staffers perform the following steps to access data on firearm-related deaths. First, staff members travel to the OCME offices in Baltimore, access the main computer, and modify the query program to specify the appropriate date range to be covered. The query then extracts all data on homicides, suicides, unintentional firearm-related deaths, and other firearm-related deaths in which the manner of death is undetermined for the period of interest. Each month, project staff receive photocopies of SHR forms from the Maryland State Police. Victim-level information on the forms is then keyed into the computer and translated into a compatible analysis format. Finally, to verify the weekly and monthly data, the surveillance system staff acquires death certificate data from the state’s Division of Health Statistics once a year. Also each year, staff members download U.S. Bureau of the Census population estimates from the Internet for the purpose of computing firearm-related death rates. Data integration. OCME and SHR data on homicides represent essentially the same events, but each source provides different characteristics of these events. For example, the OCME data include information on the number of gunshot wounds and the presence of drugs or alcohol in the body, whereas the SHR data include information on the type of weapon, victim-offender relationship, and circumstances surrounding the incident. Thus, linking the OCME and SHR data provides a more complete picture of the event than either source alone would provide. It is difficult to link records on a case-by-case basis using a computer. To do this, one needs to recode the data in the two sources so that the structures of key matching variables are the same across files. In theory, two files from the same information domain can be matched perfectly if (1) the cases in each file can be uniquely identified by a common variable or set of variables, and (2) the common identifier variables contain no errors or at least no inconsistencies between them. As a practical matter, however, common unique identifiers across official record systems do not generally exist. Even when a direct identifier such as decedent name is available in both files, the information can be (and often is) recorded with error. Unfortunately, even a small variation in an identifier (like a misspelled name) causes problems for most computer matching routines. Probabilistic record linkage procedures designed to address these problems exist,3 but we did not have access to them at the time of this study. Thus, we manually matched records. These procedures are described later, in the Evaluation Design section. Data analysis/reporting. An annual data report and monthly updates are disseminated by the Department of Health and Mental Hygiene’s Office of Public Infor-

mation, both by press release and by targeted mailings to interested individuals and organizations within the state. In addition, surveillance system staff prepare briefing materials for health officials, testify before legislative committees and task forces, and respond to other data requests (primarily from the media) as they arise. The principal use of the data is for education. The data are also being used to evaluate the effectiveness of legal policy interventions in the state. (See Sidebar Firearm-Related Deaths in Maryland: At A Glance for a summary of recent findings.)

Evaluation Design Our evaluation strategy is to link records across parallel data sources to assess the completeness of the system. The procedure is simple. We manually link OCME data with vital statistics (VS) data from death certificates. In selecting VS data as a source of comparison with OCME data, we assume that death certificates are issued and forwarded to the Division of Health Statistics for every death that occurs in the state. In other words, every OCME record of firearm-related death should have a corresponding record in the VS data. We use recordmatching to determine the empirical basis for this assumption. When records do not match, we use additional sources of information (paper copies of death certificates and police data on homicide) to explore reasons for the nonmatches. Our objective is to identify the set of true matches and the set of true nonmatches. If we can identify these sets, we have a basis for evaluating the system’s sensitivity and predictive value positive. To identify the sets, we developed a set of decision rules that we could use to match records. We used an ad hoc approach to develop these rules, based on our experience with the data. The data were sorted into lists according to manner of death, last name of decedent, first name of decedent, county of death, county of residence, gender of decedent, age of decedent, and date of death. The lists were then manually compared and the decision rules developed. We ended up assigning match status according to the following three rules (resulting in three sets of matches and nonmatches): Rule 1. (Manner of death) and (Last name of decedent) and (first name of decedent) and (county of death). Rule 2. (Manner of death) and [(Last name of decedent) or (first name of decedent)] and any 4 of the following 5 elements: [(county of residence), (county of death), (gender), (age 6 1 year), (date of death 6 1 day)]. Rule 3. (Manner of death) or [(Last name of decedent) or (first name of decedent)] and any 4 of the following 5 elements: [(county of resi-

Am J Prev Med 1998;15(3S)

49

Table 1. Firearm-related deaths by data source, manner of death, and year: Maryland Year

Sourcea

Homicide

Suicide

Manner undetermined

Unintentional

Total

1991

OCME VS Difference OCME VS Difference OCME VS Difference OCME VS Difference

390 396 26 455 461 26 481 453 128 432 420 112

266 257 19 264 249 115 271 269 12 300 293 17

12 6 16 13 15 22 11 13 22 5 8 23

4 6 22 2 5 23 4 21 217 3 11 28

672 665 17 734 730 14 767 756 111 740 732 18

1992 1993 1994

a

Difference is the Office of the Chief Medical Examiner (OCME) value minus the vital statistics (VS) value. Data from both sources are recorded according to place of occurrence (death).

dence), (county of death), (gender), (age 6 1 year), (date of death 6 1 day)] In developing Rules 2 and 3, we acknowledged that some of the key variables used in matching disagree between files but that some allowance for error is necessary to assign match status. Without allowing for some level of disagreement, the number of true matches is implausibly low because of errors. Errors in key variables can result in both false matches and false nonmatches. Thus, we must evaluate the degree of inconsistency and error in the key variables we need to link cases.

Evaluating Completeness Although medical examiner data are widely used in a number of injury-related surveillance systems, some evidence suggests that such data may under-ascertain cases and may misrepresent the character of fatal injury events.4 Given this possibility and the prominent role of medical examiner data in the Maryland system, we use two measures of completeness in case ascertainment of the surveillance system: sensitivity (the rate at which cases fail to be captured by the system, known as false negatives) and predictive value positive (the proportion of persons identified by the system as cases who actually meet the case definition, which reveals true positives as well as any false positives captured by the surveillance system). To evaluate completeness in the absence of a standard for the true number of cases, we determine the degree of overlap between multiple sources of data on firearm-related deaths in the state. Record linkage allows us to assess overlap and agreement (reliability) within linked records of key characteristics such as manner of death.

Threats to Completeness Cases enter the system by an extraction from the medical examiner’s database and the extraction is

50

based on the OCME’s classification of manner of death. The only cases entered into the system are those in which manner of death is coded as homicide, suicide, unintentional firearm-related death, or undetermined manner firearm-related death. Thus, several possible sources of error could prevent a case from being detected by the surveillance system: (1) a case of interest might not come to the attention of the medical examiner, (2) the medical examiner might misclassify the manner of death in a case of interest, (3) a properly classified case of interest might fail to be entered into the computer system, or (4) a properly classified case of interest might be entered into the computer system with the wrong manner of death code. Similar errors in the vital statistics system also are possible, with the further complexity that the person completing the death certificate does not assign the final cause-of-death classification code. That assignment is done by the state’s Division of Health Statistics on the basis of ICD protocol. That assignment is also the criterion used to select the cases for our comparison with the OCME data. Thus, errors occurring in the process of interpreting the death certificate and coding cause of death could affect the set of records available for matching. In sum, many factors could affect the overall completeness of both the OCME data and the vital statistics data. Neither data source is better or more complete than the other. No gold standard has been established.

Evaluation Results Aggregate Analysis In every year from 1991 through 1994, the annual difference recorded between the OCME and the VS ranged from 4 to 11 deaths, but the VS total never exceeded the OCME total (see Total column in Table 1). To better understand this relationship, we must

American Journal of Preventive Medicine, Volume 15, Number 3S

decompose the aggregate totals by manner of death. When we break down the totals by homicide, suicide, unintentional, and manner undetermined for 1991 to 1994 (see pertinent columns in Table 1), the pattern of results is considerably more complex than when we consider the aggregate totals only (see Total column in Table 1). For the most part, the VS data appear to include more firearm-related deaths of unintentional and undetermined manner than the OCME data, whereas the OCME tends to report greater numbers of firearm-related suicides. In 1991 and 1992, the number of VS firearm-related homicides exceeded the corresponding OCME figure, but the pattern is reversed in 1993 and 1994. In these years, OCME firearm-related homicides greatly outnumber firearm-related homicides included in the VS data. When we focus just on 1993 and 1994, we see an interesting relationship between firearm-related homicide totals and unintentional firearm-related death totals. The data suggest large and roughly offsetting differences in unintentional firearm-related deaths and firearm-related homicides between sources in these years. In any case, the clear pattern of OCME firearm-related deaths consistently outnumbering those in the VS data is not apparent when the data are subdivided by manner of death.

Results of Manual Record Matching The best way to understand these patterns is to compare each source with the other, case by case, by conducting a manual record-matching procedure. We focused our data linkage efforts on the 1993 data because they exhibit the largest differences. We applied Rules 1, 2, and 3 to data partitioned by manner of death in order to match OCME cases with VS cases. The results are as follows (Figure 2). Homicide. VS and OCME data from 1993 were linked manually on the matching criteria of name (first and last) and county of death (Rule 1). We generated lists—alphabetized by name— of firearm-related homicide victims, by county, for each data set. When comparing these lists, we matched 403 decedents and identified 128 unmatched decedents (78 from the OCME data base and 50 from VS) (see Figure 2, homicide data in column labeled Apply Rule 1). We then used relaxed criteria to compare unmatched decedents. Additional matches were defined as decedent pairs with the same first or last name and identical values for any four of the following five variables: county of residence, county of death, gender, age 6 1 year, and date of death 6 1 day (Rule 2). This procedure yielded an additional 45 matches, for a total of 448 matched pairs (see Figure 2, homicide data in column labeled Apply Rule 2). Thirty-three decedents from the OCME database and 5 from the VS database remained unmatched. Because none of the remaining

Figure 2. Matching ME and VS data on firearm-related deaths, by manner of death—Maryland, 1993.

unmatched decedents matched on more than two of the five variables, we concluded that these 38 decedents were 38 different individuals. Suicide. We used the same procedure of manually comparing sorted county and name lists to compare firearm-related suicide cases in the two data sets. After applying Rule 1 criteria of first and last name and county of death, we matched 218 pairs of decedents, leaving 108 decedents unmatched (53 in OCME and 51 in VS) (see Figure 2, suicide data in column labeled Apply Rule 1). We then relaxed the matching criteria to include first or last name and any four of the five other variables: county of residence, county of death, gender, age 6 1 year, and date of death 6 1 day (Rule 2). This procedure yielded 29 additional pairs, leaving 24 OCME decedents and 22 VS decedents unmatched. None of the remaining 46 decedents matched on more than two of the five variables, so we concluded that these were likely to be 46 different decedents. Unintentional firearm-related deaths. In the VS data, 21 gunshot victims were assigned a code for unintentional manner of death, compared with only 4 decedents in the OCME data. When we applied Rule 1, all 4 OCME cases matched a VS case, leaving 17 VS cases

Am J Prev Med 1998;15(3S)

51

Figure 3. Matching ME and VS data on all firearm-related deaths—Maryland, 1993.

unmatched (see Figure 2, unintentional death data in column labeled Apply Rule 1). Undetermined manner of death. In the 1993 OCME data, 11 decedents had codes identifying them as gunshot victims with an undetermined manner of death, compared with 12 decedents in the VS data. Ten cases from each database matched on name and county of death (see Figure 2, last row in column labeled Apply Rule 1). The remaining 3 decedents were each found in both databases but with different manners of death coded. Variation in manner of death codes. Our discovery that manner of death classification varied between data sources prompted us to develop Rule 3 (i.e., apply Rule 2, but relax the constraint on manner of death classification). We applied this rule to the entire set of firearm-related deaths (Figure 3). This resulted in 709 matches, 58 unmatched OCME cases, and 48 unmatched VS cases. After we applied Rule 3 (allowing disagreement in the classification of manner of death), the total number of matches increased by 45, for a total of 754 matches, leaving 13 unmatched OCME cases and 3 unmatched VS cases. In other words, we found 45 cases of disagreement between sources on the manner of death and 16 cases that remained unmatchable.

Clerical Review of Unmatched Cases These two sets of cases—the 16 cases that remained unmatched and the 45 cases of disagreement between sources on the manner of death—raise two questions pertinent in our evaluation of the surveillance system. First, what are the characteristics of the 16 remaining unmatched cases that cause them to be included among the set of firearm-related deaths in one data source but not the other? Second, what explains the

52

different cause of death conclusions between the OCME and VS data systems for the 45 matches identified when we applied Rule 3? To investigate the first issue, we conducted two kinds of record checks. First, we conducted computerized searches of the master injury mortality files from each data source (i.e., both firearm and nonfirearm injuryrelated deaths in both OCME and VS files) in an attempt to locate records of the previously unmatched cases. We were successful in locating 8 of the 16 unmatched cases with this procedure. All 8 cases were those that the medical examiner had concluded were firearm-related deaths but that the VS data had assigned external cause-of-death codes (E codes) classifying the cases as nonfirearm-related deaths. To further understand the cause-of-death classifications in these 16 cases, we manually searched the Division of Health Statistics’ death certificate files for paper records on these 16 individuals. We obtained photocopies of death certificates for 14 of the cases. The results of both records checks are as follows (Table 2). Unmatched OCME firearm-related cases. In all but one of the 13 unmatched OCME cases, death certificate copies supported the OCME’s manner-of-death classification. That case was a suicide involving the use of a construction tool (a nail gun). The OCME data coded it a suicide by gunshot wound while the VS E code assigned was E956, the code for suicide by cutting or piercing instrument (i.e., not firearm-related). Because a fatal injury by nail gun does not fit our definition of firearm-related death, we consider this case to have been erroneously captured by the surveillance system (a false positive). Of the remaining 12 cases, three OCME-classified firearm-related cases were given codes of E969 (late effects of intentional assault by another person) in the VS data; two OCME firearm-related cases were given codes of E958.9 (suicide by other/unspecified means) in the VS data; and another two OCME firearm-related cases were coded as E956 (suicide by cutting or piercing instrument) in the VS data. In all seven of these cases, the photocopied death certificates’ check boxes clearly indicated whether the case was a homicide, suicide, unintentional death, or death in which intent could not be determined. Further, the cause of death text fields on each of these seven certificates clearly indicated that a firearm was the underlying cause of death (i.e., agreeing with the OCME classification). We were unable to find the remaining five OCME firearm cases in computer searches of the VS injury mortality data set, despite our use of creative spellings and searches for name fragments. Our request for manual searches of the paper files resulted in death certificate copies for three of the cases, but not for the other two cases. The death certificates for each of the three cases clearly specified

American Journal of Preventive Medicine, Volume 15, Number 3S

Table 2. Reasons for and assessments of the presence or absence of unmatched cases of firearm-related death: Maryland, 1993 Number of cases

Assessment for inclusion in OCME data

E code E966 (homicide by cutting/piercing instrument) assigned to death certificate stating: “Gunshot wound of head and cutting wounds of neck.” E code E969 (late effects of intentional assault) does not distinguish firearm-related deaths from nonfirearm-related causes. E code E958.9 (suicide by other/unspecified means) assigned to death certificate stating: “Intraoral gunshot wound.” E code E958.9 (suicide by other/unspecified means) assigned; death certificate not available. E code E956 (Suicide by cutting/piercing instrument) assigned to death certificate stating: “Contact gunshot wound to chest.” E code E956 (suicide by cutting/piercing instrument) assigned to death certificate stating: “Nail gun wounds of head and chest.” Death certificates verify firearm-related causes, but record does not appear in computer file.

1

True positive

3

True positives

1

True positive

1

True positive

1

True positive

1

False positive

5 (4 suicides, 1 homicide)

True positives

Reason for OCME not matching VS

Number of cases

Assessment for inclusion in OCME data

Death certificates verify firearm-related causes, but record does not appear in computer file.

3 (suicides)

False negatives

Reason for VS not matching OCMEa

a

VS, vital statistics data; OCME, data from the Office of the Chief Medical Examiner; E code, external cause-of-death code.

deaths that were caused by gunshot wounds. We have no explanation for why these five cases do not appear in the VS data sets. Unmatched VS cases. We performed computerized searches of the master OCME database for the three unmatched VS firearm-related suicides, but without success. Photocopies of the death certificates for these cases confirmed that they were indeed firearm-related suicides. All of the cases were handled and certified by deputy medical examiners (two in Howard County and one in Frederick County), and not by the Office of Chief Medical Examiner (in Baltimore). Possibly the deputies did not send the OCME death investigation forms for these three cases; thus these cases were not entered into the OCME data base or captured by the surveillance system (false negatives). Combining the results of our case-matching efforts with the findings of these record checks, we now estimate that the total number of true firearm-related deaths in Maryland for 1993 is 769. We arrived at this total by adding the 754 matched cases to the 12 confirmed but unmatched OCME firearm-related deaths (the 13th was not confirmed), as well as the 3 confirmed but unmatched VS firearm-related deaths. This new estimate provides a basis for assessing the system’s sensitivity and positive predictive value for identifying firearm-related deaths occurring in the

state. We found the OCME data is extremely sensitive, detecting 766 of 769 (99.61%) cases. The predictive value positive is nearly perfect, with 766 of 767 (99.87%) of cases identified from the OCME data being confirmed as actual cases.

Clerical Review of Disagreements on Manner We have only suggestive evidence as to why the mannerof-death classifications disagree between the two sources in the Rule 3 matches. The available evidence is partial and not definitive, but it supports our findings that the OCME classifications are more accurate than VS E codes for the same cases. We used three strategies to gather this evidence: First, because we had access to detailed Baltimore homicide data, we examined the 38 unmatched homicides to determine whether any were reported as Baltimore City cases (Figure 2). Nineteen of the 38 cases were reported by one or both data sets as being within the jurisdiction of the Baltimore City Police Department. All 19 cases were coded as firearm-related homicides in the OCME data. In the VS data, 17 of these cases were coded with a manner of death other than homicide and the other two were coded as homicides with firearm involvement unknown. To assess which data set more accurately reported the true manner of death, we consulted the Baltimore Police Department

Am J Prev Med 1998;15(3S)

53

data set to determine the manner of death as concluded by the investigating law enforcement agency. In all 19 cases, the police department’s decision about manner of death and firearm involvement agreed with the OCME data rather than with the VS data. Second, we examined the large discrepancy in the classification of unintentional firearm-related deaths included in the VS data. Before 1993, VS data typically included 3 to 5 unintentional firearm-related deaths per year, yet 21 cases were reported in the 1993 VS data. The descriptive epidemiology of these 21 cases (overwhelmingly white males, either young adults or older adults) much more closely resembled suicide cases than unintentional gunshot victims. Deputy Chief Medical Examiner Dr. Ann Dixon reviewed the OCME case files on the 17 cases that remained unmatched, and found that none of these cases was classified by the OCME as unintentional. Third, we held a discussion with the chief nosologist in the Division of Health Statistics, Roy Tansill, to explore possible explanations for these findings. He suggested that changes in procedures in 1993 and 1994 may be responsible for many of the manner-of-death discrepancies. According to Mr. Tansill, the department switched to the use of the bottom copy of a new carbonless death certificate in 1993. Some of these copies were either illegible because the writer did not press hard enough or because handwriting was difficult to interpret. As a result, errors in E-coding the cause of death were made and the cause appeared in the data set as death due to unintentional causes—the default classification if the check boxes for homicide, suicide, or accident are not marked. According to Mr. Tansill, the procedures were revised in mid-1994 so that nosologists would be provided a photocopy of the original rather than the sometimes fuzzy carbonless copy. Our review of 1994 data (Table 1) show that the number of reported unintentional deaths was lower in 1994 than 1993, a finding consistent with Mr. Tansill’s explanation that a procedural change was implemented in mid-1994. To verify this explanation, we would need to link 1994 VS data with OCME records.

Conclusions In evaluating the Maryland surveillance system we have learned that the completeness of a firearm-related mortality surveillance system should be measured, and not assumed, because no gold standard for comparison has been established for these events (see Lessons Learned). Had we assumed the VS data were an accurate standard for comparison, we would have drawn erroneous conclusions about the sensitivity and predictive value positive of the OCME data for identifying firearm-related deaths, at least for 1993. In fact, our analysis of both the OCME and VS data shows that the

54

basic source of information for the Maryland firearmrelated mortality surveillance system appears to be highly accurate for ascertaining firearm-related deaths that occur in the state. We do not mean to imply, however, that VS data on firearm-related deaths are poor. On the contrary, the VS data in 1993 detected 757 of 769 (98.44%) cases and thus was only a little less sensitive than the OCME data. Moreover, the VS data did not include any false positives and, therefore, its predictive value positive exceeded that of the OCME data. Also, the fact that we could rely on facsimiles of the original death certificates to resolve disagreements among data sources suggests that the basis of the vital statistics data system is sound and reliable. We did find some evidence of error in the VS data’s classification of firearm-related deaths according to the ICD coding system. Our analysis suggests that the ICD E-coding system has some ambiguous definitions for firearm-related deaths. For example, deaths that result from late effects of an injury are coded so that one cannot distinguish firearm-related cases from nonfirearm-related cases. Similar classification problems may exist in other firearm-related cases that result in erroneous E-coding, as exhibited by the errors in our E956 and E958.9 cases. Our findings suggest that the overall effect of these E-coding errors in the VS data may underestimate total firearm-related deaths, although the magnitude of error is rather small. Our analysis also shows that the OCME data system is excellent. However, we recommend that the paperbased system that deputy medical examiners use to submit reports be replaced with a computerized on-line system. An on-line system would enhance both the representativeness and timeliness of data on suicides and unintentional deaths. We also conclude that manual record-matching procedures are labor-intensive, expensive, and not well suited for routine use in surveillance activities. Although we did not rigorously evaluate this aspect, we note these procedures are also subject to the quality of available information and the clerical skills and judgment of the persons doing the matching. We believe we did our best, but the task would have been quicker, cheaper, and more systematic if we could have employed computer-based probabilistic techniques to link records. Such techniques also produce results that are reproducible, reliable and more amenable to quantification of error properties. On the issue of record linkage, computer-based or otherwise, our experience underscores the importance of data that include identifiers containing sufficient discriminating power to match across sources with confidence. In most cases, the decedent’s name and county of death were sufficient to match cases. However, we encountered many cases in which name spell-

American Journal of Preventive Medicine, Volume 15, Number 3S

LESSONS LEARNED ● ●

●

Do not assume the completeness of data in a firearmrelated injury surveillance system. E-coding of firearm-related injuries may result in underreporting of firearm-related events because some E codes do not accurately distinguish firearm-related injuries from those that do not involve firearms. Pay attention to E codes for late effects of injuries. If possible, encourage the use of a common, unique identifier across data sources to increase record linkage efficiency and accuracy.

ings and occasionally the county of occurrence differed between sources. We conclude that virtually every key variable we would use for matching purposes contains measurement error. Thus, the addition of an identifier with a high level of discriminating power, such as a case report number common across data sources, would be desirable for future data linkage efforts. Having such an identifier is especially important for firearm-related mortality surveillance because no single existing data source contains all of the information one would ideally like to have in order to identify risk patterns or trends. We can enhance the utility of existing data systems if their records can be linked routinely and reliably. Manual record linkage efforts, such as those described above, are important to establish the degree to which a surveillance system is subject to false positives and false negatives when no accepted standard for comparison exists. Manual record linkages are also useful for understanding the kinds of error likely to exist in key variables used in matching and for establishing a baseline reference for evaluating the results of computer-based probabilistic linkage. Computer-based methods, and not manual record matching, will be most suitable for ongoing surveillance reporting and analysis of firearm-related injury events. Finally, this evaluation of the sensitivity and predictive value positive of the mortality data collected by the Maryland Firearm-Related Injury Surveillance System shows that the state medical examiner system is highly complete and representative in ascertaining firearmrelated deaths that occur in the state. The OCME data also appear to be valid and reliable in classifying

●

●

If a common unique identifier is unavailable, explore computer-based probabilistic techniques to link records because this method is quicker, cheaper, and more systematic; it also produces results that are more reliable, reproducible, and amenable to the detection of errors than manual record-matching procedures. Explore using existing automated data sources as a basis for surveillance before investing in new data-collection efforts that may be expensive and difficult to maintain over the long run.

firearm-related deaths according to the manner of death.

This evaluation was supported by the Centers for Disease Control and Prevention, National Center for Injury Prevention and Control (grant no. U17/CCU311058). For their help with this evaluation, we thank Denise V. Scherer, Uniform Crime Reporting Program, Maryland State Police; Shirley M. Fennell, Shirl K. Walker, and Ann Dixon, MD, Office of the Chief Medical Examiner, State of Maryland; Isabelle L. Horon, DrPH, and Roy Tansill, Division of Health Statistics, Maryland Department of Health and Mental Hygiene; and Catherine A. Gallagher, Department of Criminology and Criminal Justice, University of Maryland.

References 1. Wiersema B, Loftin C, Daub EM, Sheppard MA, Smialek JE. Firearm-related mortality in Maryland, 1976 –1996. Baltimore, Maryland: Maryland Department of Health and Mental Hygiene; 1997. 2. Fingerhut LA, Jones C, Makuc DM. Firearm and motor vehicle injury mortality-variations by state, race, and ethnicity: United States, 1990 –91. Advance data from vital and health statistics; no. 242. Hyattsville, Maryland: National Center for Health Statistics; 1994. 3. Jaro MA. Probabilistic linkage of large public health data files. Statistics in Medicine 1995; 14:491– 8. 4. Dijkhuis H, Zwerling C, Parrish G, Bennett T, Kemper HCG. Medical examiner data in injury surveillance: a comparison with death certificates. Am J Epidemiol 1994; 139:637– 43.

Am J Prev Med 1998;15(3S)

55

SIDEBAR FIREARM-RELATED DEATHS IN MARYLAND: AT A GLANCE Firearm-related injuries are the leading cause of injury deaths occurring in the State of Maryland. They exceed deaths due to motor vehicle injuries. In 1991, Maryland was one of only six states in the United States in which firearmrelated mortality equaled or exceeded motor vehicle–related injury mortality. Since 1991, the overall age-adjusted rate of deaths from firearm-related injuries has remained stable: about 15 per 100,000 residents. Young people (persons aged 15 to 24 years) have the highest risk of dying from firearm-related injuries in Maryland. The risk to this age group has increased from 29.5 deaths per 100,000 residents in 1991 to 34.1 per 100,000 in 1996. This is an increase in the death rate of nearly 17%. Males suffered 88.3% of the state’s firearm-related deaths in 1996. More than 7 males died of firearm-related injuries for every firearm-related death of a female. Among persons of all races, blacks were at greatest risk for firearm-related deaths in 1996: about 31 deaths per 100,000 black residents. In contrast, whites and other races experienced race-specific rates of 8.3 and 7.1 per 100,000 residents, respectively. Homicide accounted for 61% of the 734 firearm-related deaths in 1996. Suicide was responsible for 36%. There were only 2 unintentional firearm-related deaths (0.3%) and 12

56

firearm-related deaths for which intention had not been determined (1.6%). Among Maryland counties in 1996, firearm-related deaths were concentrated in the City of Baltimore (322 deaths) and Prince George’s County (132 deaths). Baltimore County (56 deaths), Montgomery County (40 deaths), and Anne Arundel County (35 deaths) were the third, fourth, and fifth ranking counties, respectively. Handguns, by far, are the weapon of choice in murders and non-negligent manslaughters. Approximately 91% of all firearm murders involved handguns in 1995. Handguns were used in two-thirds of all murders investigated by Maryland police agencies that year. From 1976 through 1987, the mean number of handgun murders per 100,000 residents was 4.1, but since then, the mean rate has risen to 7.1 per 100,000 residents, a 73% increase. Drug use by firearm-related homicide and suicide victims appears to have declined while alcohol consumption increased. The pattern is particularly noticeable among the homicides. The rate for firearm-related homicide victims who tested positive for drug use at autopsy is at a 4-year low. At the same time, an upturn (from 1995 to 1996) in the rate of firearm-related homicide victims who tested positive for consumption of alcohol prior to death was evident.

American Journal of Preventive Medicine, Volume 15, Number 3S