- Email: [email protected]
Concluding thoughts
Concluding thoughts
9
One of the criticisms of the discipline of firearm and toolmark identification (along with other pattern identification disciplines) has been the rigor with which it had emerged, contrasting it to DNA analysis. “If forensic individualization science had emerged from normal science, its approach and its techniques probably would resemble DNA typing, with its measurement of attributes, sampling of variation in populations, and statistical bases” (Saks and Faigman, 2008). While there might be some concern with the potential rigor with which the discipline has emerged and progressed, contrasting it to DNA analysis is not appropriate because the two disciplines are vastly different. The issue at hand is not whether or not the discipline of firearm and toolmark identification is as well defined as DNA in terms of attribute measurement, sampling of variation in populations and statistical treatment, but whether or not the discipline has emerged and grown according to the precepts of the scientific method. The first step in this process is posing a question. In the case of firearm and toolmark identification, that question was, “Is it possible to determine whether or not two or more toolmarks share a common source?” As was seen, this question was posed as early as the late 19th century when there were questions as to whether or not certain Civil War generals were killed by enemy or friendly fire. The answer was simpler in those instances because there were significant class differences in the firearms and ammunition being used by each side. However, as the potential sources opened up and there were questions as to whether a specific tool may have been a source of a toolmark, more work had to be performed. The question led to the formation of a hypothesis, whether it was specifically called that or not. That hypothesis was that due to the manufacturing processes involved in the manufacture of tools, different tools would produce different toolmarks. This hypothesis was tested in the only way possible at the time; by sampling tools, producing toolmarks and examining the marks for similarities and differences. Anticipating that the best potential for manufactured tools to have the most similarity was in consecutively manufactured tools, such tools were acquired for testing. This early testing was focused on firearms and early researchers examined barrels and other firearm parts to evaluate whether the hypothesis would hold. Based on an understanding of the manufacturing methods (remembering that such methods were few and rather rudimentary) and results of tests that early researchers performed, they became satisfied that the hypothesis was indeed valid and moved to application and training others to perform comparative analysis. In a very simplified form, what we have is the emergence of the discipline of firearm and toolmark identification according to the basic principles of the scientific method. Since those early years, other tools were examined to determine if the same principles would hold and, as manufacturing methods for firearm parts such as barrels or firing pins evolved, these newer manufacturing methods were also evaluated to determine if Firearm and Toolmark Identification. https://doi.org/10.1016/B978-0-12-813250-0.00009-7 © 2018 Elsevier Inc. All rights reserved.
156
Firearm and Toolmark Identification
the same principles would hold. Diligent practitioners within the discipline researched these issues understanding that as variables such as manufacturing methods changed, the hypothesis would have to be revisited. Experiments were designed, testing developed, and observations were made—observations that led them to conclude that the same principle of different tools will produce different toolmarks continued to hold true. The issue is that in the formative years, it was not possible to determine whether different tools would produce different toolmarks without an individual looking at the toolmarks and assessing them for similarities and differences. Considering that there was no objective standard for measuring these similarities and differences, the research was somewhat hindered. Ideally, it would have been best to be able to determine whether or not tools produced different toolmarks using an objective standard without an examiner having to view and evaluate them. However, there was no means by which this could be accomplished and, as a result, there have been concerns with respect to the development of the discipline as a whole. However, as early as the mid-1950s, efforts were made to be able to quantify the differences in striated toolmarks (Biasotti, 1959). In this way, even though there was an examiner who was evaluating the marks, at least there was an effort to quantify those differences thus lending a less subjective assessment of the observations being made. Since then, others have pursued such quantification and have also assessed its applicability to impressed toolmarks as well. In this way, there could not only be a quantifiable way to document observations, but those numbers could be evaluated to determine a reliable numerical criterion that would separate a matching condition from a nonmatching condition. Such research was promising because there was a clear quantifiable distinction between known matching and nonmatching striated toolmarks. At the same time, it was limited because it was time intensive to generate sufficient numbers to allow the work to be statistically reliable. Furthermore, the model by which quantification could be performed for striated toolmarks was pretty straightforward, and this was not as easily applied to impressed toolmarks. With the development of technology and computers, it is now finally possible to evaluate the similarity of toolmarks produced by the same and different tools without relying on an examiner to perform the actual comparison and evaluation. In this way, a number of conditions and variables can and have been tested. Not only that, but because of the processing power of computers, the numbers generated evaluating similarity of toolmarks in matching and nonmatching conditions have clear statistical significance. These studies have demonstrated that there is generally a clear separation of matching and nonmatching conditions and offer very strong support to the foundational principle underlying firearm and toolmark identification—that different tools will produce different toolmarks. Furthermore, they help to validate the previous work that, even with the human examiner element involved, those studies have value in supporting the foundational principle underlying the discipline. The question next in line then is whether or not an examiner can discern these differences so that he or she can make reliable common source determinations. The first thing to mention in this regard is that researchers who performed the computer-based studies recognized that the algorithms lacked the sophisticated ability of a trained
Concluding thoughts157
examiner. In many instances in which the computer algorithm could not make an appropriate distinction between a matching or nonmatching condition, a trained examiner could reliably make that determination. The second thing is that there is a large body of validation testing that has been done to evaluate the ability of examiners to make reliable common source determinations. While some of this validation testing has been criticized in part because of test designs and lack of blind testing, the results do strongly support that trained examiners can reliably make common source determinations. The primary obstacles with respect to reliable common source determinations include subclass characteristics and bias. The former is related to the manufacturing of the tools while bias is related to the subjective interpretation of the pattern matching process, which is at the core of comparative examination of toolmarks. Each was discussed in detail with a number of studies referenced and discussed. With respect to subclass characteristics it was recognized that, depending on the manufacturing method, the potential does exist and that they could exist on a number of tools manufactured in close sequence. At the same time, it is predictable based on the manufacturing method and it is possible to discern the manufacturing method of a particular tool by examining the tool and having an understanding of tool manufacture. In addition, strategies for evaluating tools and toolmarks were identified so that an examiner can reliably determine the potential for subclass characteristics on tool surfaces and in toolmarks. It is critical that the training of an examiner includes this very important issue because without such knowledge and training, the examination results of such an examiner could be called into question. Indeed, it could very possibly lead to an incorrect determination of common source when in fact the toolmarks were produced by a different tool. However, given proper training and education, an examiner should be able to recognize when subclass characteristics are present and take the appropriate precautions with respect to rendering interpretations of common source. Bias can have the potential to impact the interpretation of a comparative examination in firearm and toolmark identification because such interpretations are subjective in nature. There are ways in which the subjectivity can be lessened and by doing that, the potential for bias should also be alleviated accordingly. Furthermore, it is important to note that available studies have demonstrated that bias will generally not overcome a strong data set. Therefore, with respect to firearm and toolmark identification, if the pattern correspondence is strong and is not ambiguous, bias will have little to no adverse impact in the final interpretation of that data. It is of greatest concern in areas of ambiguity or a weak data set. At the same time, subjectivity can never be completely eliminated and, therefore, the potential for bias will continue to exist. Awareness and protection with respect to undue internal or external influences can aid significantly in minimizing any potential adverse impact bias might have in a case situation. Of course, the practice of firearm and toolmark identification does not take place in a vacuum. It takes place in the context of a law enforcement need to reconstruct certain elements of a crime. The eventual landing place for much this work is within the courtroom, and while the courts do not ultimately validate the science, they are charged with opining on the reliability of the work to assist the trier of fact in making
158
Firearm and Toolmark Identification
a determination of the guilt or innocence of one or more defendants. As discussed, the courts are dependent on what is presented to them and, it is essential that firearm and toolmark examiners not only understand the foundational principles with respect to the discipline but also ensure that they were properly applied and, as importantly, articulated. There were early cases in which complacency was very evident in application and articulation and, as a result, that case suffered. Furthermore, because it was believed that these issues were not isolated to those cases alone, the discipline as a whole was called into question. Since those early cases, the courts have become much more confident in the reliability of firearm and toolmark identification, though they still seem to struggle with the level of certainty with respect to a common source conclusion. Like other issues, it is essential that the examiner be able to articulate what is meant by sufficiency of agreement and what he or she means when a common source is declared. In instances in which this has been clear, the courts have generally permitted testimony to some significant level of certainty. To close, we go back to the beginning of this chapter when it was implied that pattern identification disciplines were not “normal” science and that had they been, they probably would have evolved like DNA analysis did. First and foremost, it is unknown what a “normal” science is because the use of the word would almost seem to imply that there is an abnormal science. In itself, the discipline of firearm and toolmark identification is not a science; instead it draws upon various fields of science and can very well be considered an applied science. And, as effective and important DNA analysis has been, it is not the standard for science. Rather, the standard for science lies in the scientific method itself. Due to the evolving nature of tool- manufacturing methods, which are at the very basis of the fundamental premise that different tools will produce different toolmarks, this testing according to the precepts of the scientific method has also been evolving over time. Not all of the testing was perfect. In addition, the research could be considered rather fragmented because of the sporadic nature of relevant publications. However, as gathered together in this discussion, the totality of the work demonstrates that the two foundational premises of firearm and toolmark identification are valid and have firm scientific foundations as measured by the scientific method. It is critical that this knowledge base be appreciated and understood by examiners and that they not be complacent in either the application with respect to casework or articulation when communicating with others, most specifically, the triers of fact. Finally, it is critical that research continues. The discipline is on the cusp of technology that will allow for virtual comparative microscopy. The three-dimensional imaging and processing capabilities of current technology allow a level of detail to be observed that could not be achieved using traditional comparative microscopy. The potential issue here is that our entire dataset of experiential knowledge has been acquired using traditional comparative microscopy and the transition to virtual comparative microscopy should not be assumed to be automatic. With increased data availability, it is possible that one’s concept for the levels of similarity expected to be seen in known matching and known nonmatching may be modified. It may be minimally, but the potential has to be appreciated. Furthermore, this technology may also permit an estimate of the probability of a random match based on the correspondence
Concluding thoughts159
that is being observed. This would likely help alleviate the concern of the court with respect to certainty. However, there are many variables that can impact such scores and it will be important to understand how these variables may adversely inflate these scores or adversely affect scores that should be conceivably be higher given the right conditions. The firearm and toolmark identification discipline, as others, are continually evolving. That does not mean that the current practice is not valid. It simply means that as examiners it is critical that we have a deep appreciation for the foundation that has been laid before us and a determination to improve upon that as we move forward.
References Biasotti, A., 1959. A statistical study of the individual characteristics of fired bullets. J. Forensic Sci. 4 (1), 34–50. Saks, M., Faigman, D., 2008. Failed forensics: how forensic science lost its way and how it might yet find it. Annu. Rev. Law Soc. Sci. 4, 149–171.
9
One of the criticisms of the discipline of firearm and toolmark identification (along with other pattern identification disciplines) has been the rigor with which it had emerged, contrasting it to DNA analysis. “If forensic individualization science had emerged from normal science, its approach and its techniques probably would resemble DNA typing, with its measurement of attributes, sampling of variation in populations, and statistical bases” (Saks and Faigman, 2008). While there might be some concern with the potential rigor with which the discipline has emerged and progressed, contrasting it to DNA analysis is not appropriate because the two disciplines are vastly different. The issue at hand is not whether or not the discipline of firearm and toolmark identification is as well defined as DNA in terms of attribute measurement, sampling of variation in populations and statistical treatment, but whether or not the discipline has emerged and grown according to the precepts of the scientific method. The first step in this process is posing a question. In the case of firearm and toolmark identification, that question was, “Is it possible to determine whether or not two or more toolmarks share a common source?” As was seen, this question was posed as early as the late 19th century when there were questions as to whether or not certain Civil War generals were killed by enemy or friendly fire. The answer was simpler in those instances because there were significant class differences in the firearms and ammunition being used by each side. However, as the potential sources opened up and there were questions as to whether a specific tool may have been a source of a toolmark, more work had to be performed. The question led to the formation of a hypothesis, whether it was specifically called that or not. That hypothesis was that due to the manufacturing processes involved in the manufacture of tools, different tools would produce different toolmarks. This hypothesis was tested in the only way possible at the time; by sampling tools, producing toolmarks and examining the marks for similarities and differences. Anticipating that the best potential for manufactured tools to have the most similarity was in consecutively manufactured tools, such tools were acquired for testing. This early testing was focused on firearms and early researchers examined barrels and other firearm parts to evaluate whether the hypothesis would hold. Based on an understanding of the manufacturing methods (remembering that such methods were few and rather rudimentary) and results of tests that early researchers performed, they became satisfied that the hypothesis was indeed valid and moved to application and training others to perform comparative analysis. In a very simplified form, what we have is the emergence of the discipline of firearm and toolmark identification according to the basic principles of the scientific method. Since those early years, other tools were examined to determine if the same principles would hold and, as manufacturing methods for firearm parts such as barrels or firing pins evolved, these newer manufacturing methods were also evaluated to determine if Firearm and Toolmark Identification. https://doi.org/10.1016/B978-0-12-813250-0.00009-7 © 2018 Elsevier Inc. All rights reserved.
156
Firearm and Toolmark Identification
the same principles would hold. Diligent practitioners within the discipline researched these issues understanding that as variables such as manufacturing methods changed, the hypothesis would have to be revisited. Experiments were designed, testing developed, and observations were made—observations that led them to conclude that the same principle of different tools will produce different toolmarks continued to hold true. The issue is that in the formative years, it was not possible to determine whether different tools would produce different toolmarks without an individual looking at the toolmarks and assessing them for similarities and differences. Considering that there was no objective standard for measuring these similarities and differences, the research was somewhat hindered. Ideally, it would have been best to be able to determine whether or not tools produced different toolmarks using an objective standard without an examiner having to view and evaluate them. However, there was no means by which this could be accomplished and, as a result, there have been concerns with respect to the development of the discipline as a whole. However, as early as the mid-1950s, efforts were made to be able to quantify the differences in striated toolmarks (Biasotti, 1959). In this way, even though there was an examiner who was evaluating the marks, at least there was an effort to quantify those differences thus lending a less subjective assessment of the observations being made. Since then, others have pursued such quantification and have also assessed its applicability to impressed toolmarks as well. In this way, there could not only be a quantifiable way to document observations, but those numbers could be evaluated to determine a reliable numerical criterion that would separate a matching condition from a nonmatching condition. Such research was promising because there was a clear quantifiable distinction between known matching and nonmatching striated toolmarks. At the same time, it was limited because it was time intensive to generate sufficient numbers to allow the work to be statistically reliable. Furthermore, the model by which quantification could be performed for striated toolmarks was pretty straightforward, and this was not as easily applied to impressed toolmarks. With the development of technology and computers, it is now finally possible to evaluate the similarity of toolmarks produced by the same and different tools without relying on an examiner to perform the actual comparison and evaluation. In this way, a number of conditions and variables can and have been tested. Not only that, but because of the processing power of computers, the numbers generated evaluating similarity of toolmarks in matching and nonmatching conditions have clear statistical significance. These studies have demonstrated that there is generally a clear separation of matching and nonmatching conditions and offer very strong support to the foundational principle underlying firearm and toolmark identification—that different tools will produce different toolmarks. Furthermore, they help to validate the previous work that, even with the human examiner element involved, those studies have value in supporting the foundational principle underlying the discipline. The question next in line then is whether or not an examiner can discern these differences so that he or she can make reliable common source determinations. The first thing to mention in this regard is that researchers who performed the computer-based studies recognized that the algorithms lacked the sophisticated ability of a trained
Concluding thoughts157
examiner. In many instances in which the computer algorithm could not make an appropriate distinction between a matching or nonmatching condition, a trained examiner could reliably make that determination. The second thing is that there is a large body of validation testing that has been done to evaluate the ability of examiners to make reliable common source determinations. While some of this validation testing has been criticized in part because of test designs and lack of blind testing, the results do strongly support that trained examiners can reliably make common source determinations. The primary obstacles with respect to reliable common source determinations include subclass characteristics and bias. The former is related to the manufacturing of the tools while bias is related to the subjective interpretation of the pattern matching process, which is at the core of comparative examination of toolmarks. Each was discussed in detail with a number of studies referenced and discussed. With respect to subclass characteristics it was recognized that, depending on the manufacturing method, the potential does exist and that they could exist on a number of tools manufactured in close sequence. At the same time, it is predictable based on the manufacturing method and it is possible to discern the manufacturing method of a particular tool by examining the tool and having an understanding of tool manufacture. In addition, strategies for evaluating tools and toolmarks were identified so that an examiner can reliably determine the potential for subclass characteristics on tool surfaces and in toolmarks. It is critical that the training of an examiner includes this very important issue because without such knowledge and training, the examination results of such an examiner could be called into question. Indeed, it could very possibly lead to an incorrect determination of common source when in fact the toolmarks were produced by a different tool. However, given proper training and education, an examiner should be able to recognize when subclass characteristics are present and take the appropriate precautions with respect to rendering interpretations of common source. Bias can have the potential to impact the interpretation of a comparative examination in firearm and toolmark identification because such interpretations are subjective in nature. There are ways in which the subjectivity can be lessened and by doing that, the potential for bias should also be alleviated accordingly. Furthermore, it is important to note that available studies have demonstrated that bias will generally not overcome a strong data set. Therefore, with respect to firearm and toolmark identification, if the pattern correspondence is strong and is not ambiguous, bias will have little to no adverse impact in the final interpretation of that data. It is of greatest concern in areas of ambiguity or a weak data set. At the same time, subjectivity can never be completely eliminated and, therefore, the potential for bias will continue to exist. Awareness and protection with respect to undue internal or external influences can aid significantly in minimizing any potential adverse impact bias might have in a case situation. Of course, the practice of firearm and toolmark identification does not take place in a vacuum. It takes place in the context of a law enforcement need to reconstruct certain elements of a crime. The eventual landing place for much this work is within the courtroom, and while the courts do not ultimately validate the science, they are charged with opining on the reliability of the work to assist the trier of fact in making
158
Firearm and Toolmark Identification
a determination of the guilt or innocence of one or more defendants. As discussed, the courts are dependent on what is presented to them and, it is essential that firearm and toolmark examiners not only understand the foundational principles with respect to the discipline but also ensure that they were properly applied and, as importantly, articulated. There were early cases in which complacency was very evident in application and articulation and, as a result, that case suffered. Furthermore, because it was believed that these issues were not isolated to those cases alone, the discipline as a whole was called into question. Since those early cases, the courts have become much more confident in the reliability of firearm and toolmark identification, though they still seem to struggle with the level of certainty with respect to a common source conclusion. Like other issues, it is essential that the examiner be able to articulate what is meant by sufficiency of agreement and what he or she means when a common source is declared. In instances in which this has been clear, the courts have generally permitted testimony to some significant level of certainty. To close, we go back to the beginning of this chapter when it was implied that pattern identification disciplines were not “normal” science and that had they been, they probably would have evolved like DNA analysis did. First and foremost, it is unknown what a “normal” science is because the use of the word would almost seem to imply that there is an abnormal science. In itself, the discipline of firearm and toolmark identification is not a science; instead it draws upon various fields of science and can very well be considered an applied science. And, as effective and important DNA analysis has been, it is not the standard for science. Rather, the standard for science lies in the scientific method itself. Due to the evolving nature of tool- manufacturing methods, which are at the very basis of the fundamental premise that different tools will produce different toolmarks, this testing according to the precepts of the scientific method has also been evolving over time. Not all of the testing was perfect. In addition, the research could be considered rather fragmented because of the sporadic nature of relevant publications. However, as gathered together in this discussion, the totality of the work demonstrates that the two foundational premises of firearm and toolmark identification are valid and have firm scientific foundations as measured by the scientific method. It is critical that this knowledge base be appreciated and understood by examiners and that they not be complacent in either the application with respect to casework or articulation when communicating with others, most specifically, the triers of fact. Finally, it is critical that research continues. The discipline is on the cusp of technology that will allow for virtual comparative microscopy. The three-dimensional imaging and processing capabilities of current technology allow a level of detail to be observed that could not be achieved using traditional comparative microscopy. The potential issue here is that our entire dataset of experiential knowledge has been acquired using traditional comparative microscopy and the transition to virtual comparative microscopy should not be assumed to be automatic. With increased data availability, it is possible that one’s concept for the levels of similarity expected to be seen in known matching and known nonmatching may be modified. It may be minimally, but the potential has to be appreciated. Furthermore, this technology may also permit an estimate of the probability of a random match based on the correspondence
Concluding thoughts159
that is being observed. This would likely help alleviate the concern of the court with respect to certainty. However, there are many variables that can impact such scores and it will be important to understand how these variables may adversely inflate these scores or adversely affect scores that should be conceivably be higher given the right conditions. The firearm and toolmark identification discipline, as others, are continually evolving. That does not mean that the current practice is not valid. It simply means that as examiners it is critical that we have a deep appreciation for the foundation that has been laid before us and a determination to improve upon that as we move forward.
References Biasotti, A., 1959. A statistical study of the individual characteristics of fired bullets. J. Forensic Sci. 4 (1), 34–50. Saks, M., Faigman, D., 2008. Failed forensics: how forensic science lost its way and how it might yet find it. Annu. Rev. Law Soc. Sci. 4, 149–171.