- Email: [email protected]
Contextual information management: An example of independent-checking in the review of laboratory-based bloodstain pattern analysis
Accepted Manuscript Contextual information independent-checking in bloodstain pattern analysis
management: An example of the review of laboratory-based
Nikola K.P. Osborne, Michael C. Taylor PII: DOI: Reference:
S1355-0306(18)30006-6 doi:10.1016/j.scijus.2018.01.001 SCIJUS 715
To appear in:
Science & Justice
Received date: Revised date: Accepted date:
5 June 2017 9 January 2018 12 January 2018
Please cite this article as: Nikola K.P. Osborne, Michael C. Taylor , Contextual information management: An example of independent-checking in the review of laboratory-based bloodstain pattern analysis. The address for the corresponding author was captured as affiliation for all authors. Please check if appropriate. Scijus(2018), doi:10.1016/ j.scijus.2018.01.001
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Contextual Information Management: An example of independent-checking in the review of laboratory-based Bloodstain Pattern Analysis Paper prepared for submission to Science & Justice Nikola KP Osbornea*1 & Michael C Taylorb a
PT
The Institute of Environmental Science & Research (ESR), PO Box 29181, Christchurch 8540, New Zealand; [email protected] b
SC
RI
The Institute of Environmental Science & Research (ESR), PO Box 29181, Christchurch 8540, New Zealand; [email protected]
AC C
EP T
ED
MA
NU
*Corresponding author.
1
Please note, Dr. Osborne is now based at the University of California Irvine, Department of Criminology, Law, and Society, 2340 Social Ecology II, Irvine, CA 92697-7080
ACCEPTED MANUSCRIPT Abstract This article describes a New Zealand forensic agency’s contextual information management protocol for bloodstain pattern evidence examined in the laboratory. In an effort to create a protocol that would have minimal impact on current work-flow, while still effectively removing task-irrelevant contextual information, the protocol was designed following an in-
PT
depth consultation with management and forensic staff. The resulting design was for a
RI
protocol of independent-checking (i.e. blind peer-review) where the checker’s interpretation
SC
of the evidence is conducted in the absence of case information and the original examiner’s notes or interpretation(s). At the conclusion of a ten-case trial period, there was widespread
NU
agreement that the protocol had minimal impact on the number of people required, the cost, or the time to complete an item examination. The agency is now looking to adopt the protocol
MA
into standard operating procedures and in some cases the protocol has been extended to cover other laboratory-based examinations (e.g. fabric damage, shoeprint examination, and physical
ED
fits). The protocol developed during this trial provides a useful example for agencies seeking
EP T
to adopt contextual information management into their workflow.
AC C
Keywords: Forensic science, bloodstain pattern analysis, cognitive bias, blind peer-review, independent check, task-relevant information.
ACCEPTED MANUSCRIPT 1. Introduction The past decade has seen an increase in discussion [e.g. ,1–13] and research [e.g. ,14–25] relating to the issue of contextual bias in forensic decision-making. The main concern highlighted by commentators is that information irrelevant to a forensic examiner’s task may unconsciously influence (or bias) the examiner’s opinion. Indeed, the recent US President’s
PT
Council of Advisors on Science and Technology (PCAST) report [11] released the following
RI
call: “because contextual bias may have an impact on experts’ decisions, courts should assess the measures taken to mitigate bias during casework” [11 p. 100]. Such statements make it
SC
increasingly difficult for forensic agencies and practitioners to simply ignore the potential for
NU
extraneous contextual information to have a biasing influence on their opinions. At the forefront of recommendations to address the issue of contextual bias is a requirement
MA
to have examiners adopt contextual information management (CIM) procedures [1,6,9,12,13,26]. The main objective of CIM is to shield examiners from potentially biasing
ED
information, and that which is not relevant to their task, while still allowing access to task-
EP T
relevant information. The main CIM approaches discussed in the literature include: 1) (linear) sequential unmasking [5,27], 2) the context-manager model [26,28,29], and 3) blind peer-
AC C
review [12,30].
Sequential unmasking [5] and linear sequential unmasking [27] are recommended for sourcelevel comparison disciplines (e.g., DNA, fingerprints, and shoeprint comparisons), and specify the optimal order in which to examine the forensic material. Forensic experts should examine the latent (i.e. crime-scene) material prior to being exposed to the reference (i.e. suspect/known) material. In this way, they are working from the evidence to the suspect. Linear sequential unmasking also requires examiners to state levels of confidence in their opinion regarding the material under examination [27]. This method allows for flexibility in the decision-making process, but it also requires that the examiner be transparent in how s/he
ACCEPTED MANUSCRIPT has incorporated contextual information into his/her analysis, and what influence this information has had. The context-manager model, is an alternative approach whereby a context-manager has access to all of the contextual information, but only passes on to the examiner that which is relevant for the examination. In this way, the examiner conducts his/her examination in the
PT
absence of potentially biasing information, yet someone (i.e. the context-manager) still has
RI
access to the contextual information in order to determine the type of examination that might
SC
be necessary, and to put the examination and subsequent results in context with the case as a whole. This approach has been reported in forensic document [28] and firearm [26,29]
NU
examinations.
MA
Peer-review—an assessment or review of an examiner’s opinion and case notes—is an opportunity for any errors to be “picked up” prior to the case report being finalised. While
ED
many laboratories routinely perform a technical2 or administrative3 review of cases, an independent reassessment (from here on in, referred to as independent-check) of each
EP T
individual item examined is not so commonplace [30]. Independent checks are when the checker does not have access to the case information or the
AC C
original examiner’s conclusions. If a checker is not shielded from the same potentially biasing case information, it is possible that s/he will experience the same bias as the original examiner. In addition, knowledge of the previous examiner’s conclusion may introduce an additional source of bias. In fingerprint interpretation, for example, the linear ACE-V (assessment, comparison, evaluation – verification) methodology has been criticised for creating an expectation that a latent and comparison print share the same source [31,32]. That 2
A technical review is the review of the case notes, report, and conclusions. This is performed by another scientist. 3 An administrative review is to check for typographical and other non-examination-related errors. This does not necessarily have to be completed by a scientist.
ACCEPTED MANUSCRIPT is because only opinions which indicate a same-source conclusion are reviewed. Some forensic agencies have now adopted a methodology whereby all opinions are reviewed (including those that suggest exclusion or are inconclusive), and this is done in way that means the reviewer is not aware of the original opinion [33]. Bloodstain Pattern Analysis (BPA) is the analysis of the size, shape, and distribution of
PT
bloodstains. The methods for applying CIM in BPA are less clear than in other pattern
RI
comparison or source-conclusion forensic disciplines. BPA generally requires analysts to
SC
determine a mechanistic cause of the pattern (i.e. activity level opinion), and to put the bloodstain patterns in context with other elements of the scene. In these cases, some
NU
contextual information might be highly relevant – even if it is potentially biasing.
MA
Indeed the provision of contextual information has been identified as a potential source of error in BPA, with two systematic studies [14,15] showing that negatively-biasing case
ED
information (i.e. information suggesting an incorrect pattern type) increased the rate of incorrect pattern classifications compared to neutral or positively biasing case information
EP T
(i.e. information suggesting the ground truth pattern type). These studies did not include a condition that resembled a peer-review process, and therefore it is not clear whether or not a
AC C
second examiner would have identified and corrected these errors. It is clear, however, that bloodstain pattern analysts should recognize and address the potential for contextual information to influence their opinions. A New Zealand forensic agency, the Institute of Environmental Science and Research (ESR)4 , is actively endeavouring to minimise the potential for bias in their forensic examinations. This paper describes the development and trial of a contextual information
4
ESR is an independent government agency that is the sole provider of forensic services to the New Zealand Police. The Forensic Business Group comprises three Service Centres, located in Auckland, Kenepuru, and Christchurch.
ACCEPTED MANUSCRIPT management protocol for BPA within this agency. We present feedback after a trial period of ten cases. 2. Method 2.1 Protocol Development The first step in developing a protocol was to understand current procedures within each of
PT
ESR’s three forensic laboratories. We interviewed a cross section of management and
RI
scientific staff about: 1) current procedures for the examination of bloodstains at a crime
SC
scene and for bloodstained items examined in the laboratory, 2) the relevance of various types of information in BPA, and 3) the feasibility and practicality of introducing a context
NU
management system into current procedures.
MA
All those interviewed recognised that bias could be an issue in forensic examinations, however, there were concerns that extreme context management (i.e. the removal of all case information) could have a negative impact on their ability to effectively conduct a scene
ED
examination. In particular, scientists identified the need for basic case information in order to
EP T
determine the purpose of an examination (e.g. to inform which resource kits to take to the scene and the types of examinations required to determine and answer the investigative questions). For items examined in the laboratory, however, interviewees reported that the
AC C
need for case information is largely limited to identifying the case in the Laboratory Information Management System (LIMS). Although case information may not be needed for the item examination per se, the case manager who attends the scene generally also conducts the laboratory examinations for that case. The case manager, therefore, has the case information as a result of the scene examination. Efforts to have items obtained from a scene examination sent to an alternative laboratory, where a scientist who has no knowledge of the case circumstances can perform the item examination, were deemed possible but impractical.
ACCEPTED MANUSCRIPT Based on the interviews and examination of work-flow, we determined that an independentcheck (sometimes referred to as a blind peer-review) was the most appropriate and practical context management system to adopt for laboratory-based item examinations in BPA at ESR. ESR routinely uses a review process which includes review of the case notes and report, and can also include re-examination of an item (i.e. a bench peer-check). Currently, this bench
PT
peer-check is not always independent (i.e. blinded). In the proposed methodology, items are
RI
still to be examined by the case manager; however, a non-scene-attending scientist from the
SC
same laboratory performs a separate, independent analysis without any of the potential bias that could result from conducting the scene examination or from knowledge of the case
NU
manager’s opinion(s). 2.2 Trial Procedure
MA
Before deciding whether or not to recommend independent-checking as a standard operating procedure, we conducted a trial to assess the impact of this CIM protocol. The trial was
ED
conducted over a twelve week period during which the CIM protocol was applied to ten cases
EP T
that were received across ESR’s three laboratories as part of their routine casework. For the trial period, cases included in the trial were those in which bloodstain pattern analysis
AC C
was required on items submitted to the laboratory. These could include items that came from a scene that ESR attended, or items that were submitted by police but ESR did not attend the scene. Cases not included in the trial were those where the items requiring examination were sampled for DNA analysis but the examiner did not conduct an interpretation of any bloodstaining. The steps for the independent check were: 2.3 Independent-Check Protocol 1. Case manager (CM) assigned to the case.
ACCEPTED MANUSCRIPT 2. CM assigns an independent-checker (IC). IC informed that information regarding this case should be avoided where possible.5 3. Senior technician (ST) or CM draws a diagram of the item. ST screens and samples for blood, identifying areas of staining and recording the results of any tests performed. This diagram is replicated before any bloodstain pattern interpretations are made. The original is passed to the CM and the copy is put aside for the IC (or vice versa6 ).
PT
4. CM carries out BPA on the item(s) as per standard procedures, thoroughly documents all interpretations, and provides evidence–based conclusions.
RI
5. CM’s interpretations are omitted from any material that is to be passed to the IC.
SC
6. All case information is also omitted from material to be passed on to the IC. 7. IC conducts BPA on the item(s). IC thoroughly documents all interpretations on the 7
NU
diagram provided, and provides evidence-based conclusions.
8. CM compares IC’s interpretations with his/her interpretations. 9. If any inconsistencies between CM’s and IC’s interpretations arise, the standard
MA
laboratory procedure to resolve inconsistencies is followed. 10. Technical review completed by IC or additional scientist after context-managed
2.4 Information Management
ED
interpretations are made.8
was available:
Results of tests for blood Areas of staining identified by the CM/ST Location of bloodstaining on the item and location of other bloodstains in the scene. Location of the item in the scene
AC C
EP T
During the trial, the independent-checker could have access to the following information, if it
The following information was not made available to the independent-checker: 5
IC was not always decided this early in the procedure. He or she might have been assigned later depending on availability of scientists at the time. As a matter of course, however, only case managers and other sceneattending staff are aware of the case circumstances. 6 Because the IC does not have access to the CM’s notes and interpretations but only the diagram and the item itself, it is possible for the IC to conduct his/her examination first. The order of examination (i.e., whether CM or IC does his/her examination first) is not important; what is important is that the person conducting the second examination is not aware of the first examiner’s conclusions. 7 We did not stipulate a time frame for analysts to complete this, or any other step. Analysts were to proceed as they usually would, taking into account case load, urgency, and any other factors outside of our control. 8 If the IC is also the technical reviewer then by virtue of his/her role in determining if the case notes support the reported conclusions, he/she will be exposed to the contextual information and CM’s opinions in this step.
ACCEPTED MANUSCRIPT
CM’s interpretations Type of case Operation name Police briefing/Case circumstances Names of persons involved Alleged activities within the scene Police theory Witness statements Medical findings Other forensic reports
PT
In addition, informal discussion between peers about the case was deliberately limited. Upon
RI
completion of each case examination and independent check, case and item examination
SC
notes and interpretations were passed to the researchers. The trial was over once all 10 steps
NU
were completed on 10 cases that met the inclusion criteria. 3. Results
MA
As this was a trial conducted in the course of actual forensic casework, we could not assess examiners’ accuracy (i.e. we did not know the ground truth). We were able to observe,
ED
however, whether or not the independent-check process resulted in any disagreements or inconsistencies between the case manager’s and independent-checker’s opinions, and assess
EP T
if and how these were resolved. In nine out of the ten cases, there were no inconsistencies between the case manager’s and the independent checker’s interpretations. In one of the ten
AC C
cases, a pattern type identified as possible by the case manager, was not included (although not excluded) as a possible pattern type by the checker. This case and how the inconsistency was resolved is described below. 3.1 Case Example 3.1.1. Contextual information In this case, the case manager attended the scene and was aware of the following information:
Male A had seen a photo of his ex-partner with another man and became upset.
Male A arrived at ex partner’s address with a kitchen knife concealed in his pocket.
Male A punched the glass ranch slider, smashing it, cutting his wrist.
ACCEPTED MANUSCRIPT
Male A grabbed ex-partner and another occupant, pushing them into the living room.
Male A kicked and pushed occupants.
Three occupants wrestled Male A to the ground before he could use the knife on them.
3.1.2. Items submitted for examination One pair of trouser pants recovered from Male A was submitted to the laboratory for
PT
examination for blood and bloodstain pattern analysis. 3.1.3. Purpose of examination
RI
The stated purpose of this examination was to determine the cause of any bloodstaining on
SC
Male A’s trouser pants.
NU
3.1.4. Interpretation results
The following is a summary of the case manager’s interpretation notes (bloodstain pattern
MA
classification underlined):
“Airborne blood generated from force/impact or from blood into blood near lower R
ED
leg”
“Nothing in these stains to indicate cough/sneeze (expirated) is likely”
EP T
The following is a summary of the independent checker’s interpretation notes (bloodstain pattern classification underlined):
“Collection of fine spots (< 1mm) on lower shin/ankle at front of right leg = Close
AC C
proximity to impact of exposed source of blood” 3.1.4 Resolution
In this example, the case manager had indicated blood dripping into blood and impact as possible mechanisms for the blood deposition on the lower right leg of the trousers. The independent-checker had only included impact in his/her assessment. S/he neither included nor excluded blood dripping into blood as a possible account for the pattern. While these differing pattern mechanism accounts are not a disagreement per se, the inconsistency warranted further investigation. A second independent-check from an additional scientist also
ACCEPTED MANUSCRIPT resulted in an opinion that the pattern could be the result of impact or blood dripping into blood. Furthermore, several experiments were conducted in an attempt to replicate the patterns observed on the trousers. Using mechanisms that could create both blood dripping into blood and impact patterns, the case manager and independent-checker determined that either of these pattern mechanisms could account for the observed pattern on the trousers.
PT
The results of the experimentation and third examination meant that both impact and blood
RI
dripping into blood were included as mechanisms in the final report. The case manager
SC
reflected that the knowledge of Male A’s injury (cut wrist) meant that s/he considered whether the blood could have dripped onto blood present on the floor (or another surface
NU
close to the cuff of the trousers), and resulted in satellite spatter from blood dripping into
3.2 Summary of remaining cases
MA
blood.
The remaining cases included items of bloodstained clothing, shoes, and wood. Across these
ED
cases, analysts identified various pattern classification types including transfer, spatter, and
EP T
passive staining.
3.3 Analysts’ response to the trial
Overall, those involved in the trial responded positively to the introduction of an
AC C
independent-check system. Feedback from the scientists and technicians involved in the first ten cases included:
“[The method] is minimal extra hassle and time, [we] will continue the process as standard practice from now on.” - Senior Scientist “[The method] takes slightly longer as the reviewer has to write their own notes from scratch. However, it ensures that a true blind peer-check occurs.” – Senior Scientist “We will continue to use it, and should use it for other evidence types that require bench review (i.e. fabric damage, footwear comparisons, and physical fits).” – Senior Scientist
ACCEPTED MANUSCRIPT “At the technician level, the only added task is to make an extra copy of the diagram. My opinion is that it should continue onwards.” – Senior Technician 4. Discussion This paper describes the trial of a methodology of contextual information management in laboratory-based bloodstain pattern analysis, where a second assessment of the evidence
PT
occurred in the absence of any contextual case information or the original examiner’s notes
RI
and opinion(s). This trial was conducted to assess the feasibility of an independent-check protocol in practice. There were no reports of adverse consequences of the protocol, and the
SC
examiners who participated reported that they appreciated what it was trying to achieve and
NU
that it gave them more confidence that their opinions were unbiased. Introducing the protocol as standard practice, therefore, appears to be both achievable and welcomed.
MA
In nine out of ten cases, both the case manager and independent-checker reached the same opinion. In the one case where the case manager and independent-checker proffered
ED
inconsistent opinions; the case manager, by his/her own admission, was influenced by the
EP T
case circumstances and indicated blood dripping into blood as a possible mechanism because of these circumstances, not just because of the pattern characteristics. While this could be construed as bias, we could also contend that the contextual information has in fact assisted
AC C
the case manager to form a relevant hypothesis. The value of the independent-check here, therefore, is that it has identified a potentially biased opinion and has forced steps to reconsider the opinion in an unbiased way. The fact that this reconsideration—second independent-check and experimentation—supported the original (possibly biased) opinion in no way justifies the automatic inclusion of contextual information in BPA; rather, it points to the need to carefully manage its introduction. Furthermore, aside from raising questions about the reliability of the first independentchecker’s opinion (i.e. if the observations alone supported blood dripping into blood, then the
ACCEPTED MANUSCRIPT checker should have included this pattern mechanism), this inconsistency highlights at least two important issues for consideration in the management of contextual information in BPA, and for understanding BPA methodology in general. First, it appears that there may be a disadvantage to relying solely on blind examinations (i.e. where no examiner has access to contextual information at any point). Because contextual
PT
information may help an examiner to consider possible hypotheses to test, an absence of this
RI
information could result in a failure of abductive reasoning. Conversely, however, if all
SC
examinations of the same evidence are conducted with contextual information (i.e. check not independent), then any undue influence of the contextual information might never be
NU
recognized. An independent-check, therefore, helps to strike the delicate balance between
MA
having too much information and too little information.
Second, the inconsistency gives us a deeper understanding of the judgemental process in BPA. In this example, the case manager has used the contextual information to render a
ED
reconstructive theory that the blood could have dripped from the trouser-wearer’s injured
EP T
hand and onto the floor and then back onto the cuff of the trousers, thus creating a pattern resulting from blood dripping into blood. The reconstructive theory, therefore, has informed
AC C
the classification (abductive reasoning), rather than the classification informing the reconstructive theory. It is clear that BPA involves complex reasoning processes, and while contextual information may be biasing, it may also have a part to play in the development of relevant hypotheses to test. Here, we recognize that current bloodstain pattern classification methods and terminology do not offer a clear distinction between opinions related to an analyst’s observations (i.e. pattern characterization) and those related to his/her interpretations (i.e. mechanism determination, scene reconstruction). As such, understanding the relevance of contextual information, and how to manage it for each task, is not a simple endeavour in BPA. Until bloodstain pattern
ACCEPTED MANUSCRIPT classification methods and terminology clearly separate the currently overlapping components of observation and interpretation [34], we offer that an independent-check can help to disentangle the relative contribution of contextual information versus the contribution of the observations to the examiner’s opinion. This disentanglement is of great importance if the relevance or accuracy of the contextual information upon which an examination has relied
PT
upon is contested, or if the observations alone do not support the proffered opinion.
RI
As this is a relatively new methodology, we may need to consider making some changes and
SC
adjustments as difficult or complex cases arise. Complex cases might require discussion between two examiners prior to a conclusion being reported. Because complexity and
NU
ambiguity increases the potential for extraneous information to influence decision-making [1,12,35–37], complex examinations are most at risk of resulting in biased conclusions. In
MA
these cases, the case manager could consider sending the item to another examiner outside of his/her laboratory. Furthermore, examiners are likely to profit from engaging one another in
ED
discussions regarding how they have addressed context management for difficult or unusual
EP T
cases.
Although the procedure was originally designed for the examination of bloodstained items, it
AC C
has the potential to benefit all forensic disciplines. Indeed, scientists in this trial reported that they began to include other evidence types (e.g. shoeprint comparison, fabric damage and physical fits) even though they were not prompted to do so for the purpose of our trial. An additional benefit of this procedure is that it could allow for blind testing of examiner proficiency. That is, items for which the ground-truth is known, could be presented to the checkers as though they were case items. Testing in this way provides a more realistic assessment of proficiency than tests in which the examiner knows that he or she is being tested. The PCAST report [11] acknowledges the difficulties associated with implementing
ACCEPTED MANUSCRIPT blind proficiency testing, however, it recognizes the advantages and expects that all forensic laboratories include blind-testing into their routine work-flow within the next five years. Although we have presented a practical methodology for managing contextual information in the laboratory, we have not yet addressed the more difficult task of managing extraneous information at a crime scene. Scene-going forensic examiners cannot simply operate in a
PT
vacuum; the (potential) circumstances of the crime are inherent in the scene, after all that is
RI
what the examiner is there to assess. Furthermore, analysts are usually under (both implicit
SC
and explicit) pressure to examine the scene quickly and efficiently. For many forensic agencies, it is neither practical nor efficient to have two analysts perform independent scene
NU
examinations. Although checkers can conduct their BPA review through photographs and diagrams, these do not provide the same spatial representation that a scene does. The ability
MA
to capture a 3D representation of the scene may go some way in providing the opportunity for a second analyst to “revisit” the scene blind to contextual information and the original
ED
examiner’s opinions.
EP T
There are a number of additional points worth making in regards to the implementation of contextual information management for bloodstain pattern analysis: Contextual information regarding high profile cases may be difficult to avoid,
AC C
particularly in places where homicide rates are relatively low and case information is widely publicised.
Bloodstain pattern evidence, by its very nature, creates an expectation or insinuation that violence or pain has occurred. Furthermore, it is often only the ‘serious’ cases that forensic services are requested for. It is unclear to what extent this inherent and unavoidable contextual information could play a role in analysts’ decision-making.
At times there may only be a small number of analysts with an agency who are able to perform a BPA examination. If all available analysts have attended the scene (and therefore are aware of the case circumstances) it might be appropriate to send items to another laboratory to be reviewed.
ACCEPTED MANUSCRIPT
In laboratories where scientists often perform examinations across multiple forensic disciplines, there is a chance that checkers will have worked on other aspects of the case and therefore know the case circumstances. Ideally, items should be sent to an alternative laboratory to be checked by a scientist blind to the case circumstances, but at a minimum the check should always be performed blind to the original examiner’s interpretation, and examiners should avoid discussing the results of other forensic
PT
examinations. 4.1 Conclusions
RI
We have developed and trialled a simple and practical contextual information management
SC
procedure for laboratory-based bloodstain pattern analysis. In nine of the ten trial cases, both the case manager and independent checker reached the same opinion. In the single case in
NU
which there was a difference of opinion, this was resolved by enlisting a second independent
MA
checker and through experimentation.
Adherence to the described procedure will decrease the potential for extraneous contextual
ED
information to influence the independent checker’s opinion. Furthermore, assessing any difference between the case manager’s and the independent checker’s opinion allows for
EP T
some understanding of the relative contribution of the contextual information to the contextually- informed opinion. This understanding should ultimately lead to better-informed
AC C
and considered determinations.
This procedure also allows the courts to assess that the agency has taken steps to shield their analysts from potentially biasing information. Given the results of this trial and the positive response from participants to the independent-check protocol, the acceptance of such a protocol into standard operating procedures appears to be both achievable and worthwhile. The protocol developed during this trial provides a useful example for agencies seeking to adopt contextual information management into their workflow. Acknowledgements
ACCEPTED MANUSCRIPT This project was funded by a grant from the National Institute of Justice (NIJ #2014-DN-BXK032). The authors gratefully acknowledge the forensic staff and management at The
AC C
EP T
ED
MA
NU
SC
RI
PT
Institute of Environmental Science and Research (ESR) for their participation in the trial.
ACCEPTED MANUSCRIPT
5. References [1]
S.M. Kassin, I.E. Dror, J. Kukucka, The forensic confirmation bias: Problems, perspectives, and proposed solutions, J. Appl. Res. Mem. Cogn. 2 (2013) 42–52.
[2]
PT
doi:10.1016/j.jarmac.2013.01.001. I.E. Dror, The ambition to be scientific: Human expert performance and objectivity,
I.E. Dror, Cognitive forensics and experimental research about bias in forensic
SC
[3]
RI
Sci. Justice. 53 (2013) 81–82. doi:10.1016/j.scijus.2013.03.002.
[4]
NU
casework, Sci. Justice. 52 (2012) 128–130. doi:10.1016/j.scijus.2012.03.006. R. Zajac, N.K.P. Osborne, L. Singley, M.C. Taylor, Contextual Bias: What Bloodstain
[5]
MA
Pattern Analysts Need to Know, J. Bloodstain Pattern Anal. 31 (2005). D.E. Krane, S. Ford, J.R. Gilder, K. Inman, A. Jamieson, R. Koppl, I.L. Kornfield,
ED
M.D. Risinger, N. Rudin, M.S. Taylor, W.C. Thompson, Sequential unmasking: A
EP T
means of minimizing observer effects in forensic DNA interpretation, J. Forensic Sci. 53 (2008) 1006–1007. doi:10.1111/j.1556-4029.2008.00787.x. G. Edmond, A. Towler, B. Growns, G. Ribeiro, B. Found, D. White, K. Ballantyne,
AC C
[6]
R.A. Searston, M.B. Thompson, J.M. Tangen, R.I. Kemp, K. Matire, Thinking forensics: Cognitive science for forensic practitioners, Sci. Justice. in press (2016) 1– 27. doi:http://dx.doi.org/10.1016/j.scijus.2016.11.005. [7]
R.D. Stoel, I.E. Dror, L.S. Miller, Bias among forensic document examiners: Still a need for procedural changes, Aust. J. Forensic Sci. 46 (2014) 91–97. doi:10.1080/00450618.2013.797026.
[8]
B. Found, Deciphering the human condition: the rise of cognitive forensics, Aust. J.
ACCEPTED MANUSCRIPT Forensic Sci. 47 (2015) 386–401. doi:10.1080/00450618.2014.965204. [9]
S.A. Cole, Implementing counter-measures against confirmation bias in forensic science, J. Appl. Res. Mem. Cogn. 2 (2013) 61–62. doi:10.1016/j.jarmac.2013.01.011.
[10]
National Research Council, Strengthening Forensic Science in the United States: A
[11]
PT
Path Forward, The National Academies Press, Washington, D.C., 2009. President’s Council of Advisors on Science and Technology, Forensic Science in
M.D. Risinger, M.J. Saks, W.C. Thompson, R. Rosenthal, The Daubert/Kumho
SC
[12]
RI
Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, 2016.
NU
Implications of Expectation and Suggestion Observer Effects in Forensic Science: Hidden Effects of Expectation and Suggestion, Calif. Law Rev. 90 (2002) 1–56. W.C. Thompson, What role should investigative facts play in the evaluation of
MA
[13]
scientific evidence?, Aust. J. Forensic Sci. 43 (2011) 123–134.
M.C. Taylor, T.L. Laber, P.E. Kish, G. Owens, N.K.P. Osborne, The Reliability of
EP T
[14]
ED
doi:10.1080/00450618.2010.541499.
Pattern Classification in Bloodstain Pattern Analysis, Part 1: Bloodstain Patterns on Rigid Non-absorbent Surfaces, J. Forensic Sci. 61 (2016) 922–927. doi:10.1111/1556-
[15]
AC C
4029.13091.
M.C. Taylor, T.L. Laber, P.E. Kish, G. Owens, N.K.P. Osborne, The Reliability of Pattern Classification in Bloodstain Pattern Analysis—Part 2: Bloodstain Patterns on Fabric Surfaces, J. Forensic Sci. 61 (2016) 1461–1466. doi:10.1111/1556-4029.13191.
[16]
I.E. Dror, D. Charlton, A.E. Péron, Contextual information renders experts vulnerable to making erroneous identifications, Forensic Sci. Int. 156 (2006) 74–78. doi:10.1016/j.forsciint.2005.10.017.
ACCEPTED MANUSCRIPT [17]
S. Nakhaeizadeh, I.E. Dror, R.M. Morgan, Cognitive bias in forensic anthropology: Visual assessment of skeletal remains is susceptible to confirmation bias, Sci. Justice. 54 (2014) 208–214. doi:10.1016/j.scijus.2013.11.003.
[18]
I.E. Dror, D. Charlton, Why experts make errors, J. Forensic Identif. 56 (2006) 600–
[19]
PT
616. J. Kukucka, S.M. Kassin, Do confessions taint perceptions of handwriting evidence?
RI
An empirical test of the forensic confirmation bias, Law Hum. Behav. 38 (2014) 256–
N.K.P. Osborne, M.C. Taylor, R. Zajac, Exploring the role of contextual information
NU
[20]
SC
270. doi:10.1037/lhb0000066.
in bloodstain pattern analysis: A qualitative approach, Forensic Sci. Int. 260 (2016) 1–
[21]
MA
8. doi:10.1016/j.forsciint.2015.12.039.
N.K.P. Osborne, M.C. Taylor, M. Healey, R. Zajac, Bloodstain pattern classification:
ED
Accuracy, effect of contextual information and the role of analyst characteristics, Sci.
[22]
EP T
Justice. 56 (2016) 123–128. doi:10.1016/j.scijus.2015.12.005. N.K.P. Osborne, S. Woods, J. Kieser, R. Zajac, Does contextual information bias bitemark comparisons?, Sci. Justice. 54 (2014) 267–273.
[23]
AC C
doi:10.1016/j.scijus.2013.12.005. J.H. Kerstholt, A. Eikelboom, T. Dijkman, R. Stoel, R. Hermsen, B. van Leuven, Does suggestive information cause a confirmation bias in bullet comparisons?, Forensic Sci. Int. 198 (2010) 138–142. doi:10.1016/j.forsciint.2010.02.007. [24]
J.H. Kerstholt, R. Paashuis, M. Sjerps, Shoe print examinations: Effects of expectation, complexity and experience, Forensic Sci. Int. 165 (2007) 30–34. doi:10.1016/j.forsciint.2006.02.039.
[25]
L.J. Hall, E. Player, Will the introduction of an emotional context affect fingerprint
ACCEPTED MANUSCRIPT analysis and decision-making?, Forensic Sci. Int. 181 (2008) 36–39. doi:10.1016/j.forsciint.2008.08.008. [26]
E.J.A.T. Mattijssen, W. Kerkhoff, C.E.H. Berger, I.E. Dror, R.D. Stoel, Implementing context information management in forensic casework: Minimizing contextual bias in firearms examination, Sci. Justice. 56 (2016) 113–122.
I.E. Dror, W.C. Thompson, C.A. Meissner, I. Kornfield, D.E. Krane, M.J. Saks, M.
RI
[27]
PT
doi:10.1016/j.scijus.2015.11.004.
SC
Risinger, Letter to the Editor- Context Management Toolbox: A Linear Sequential Unmasking (LSU) approach for mimimizing cognitive bias in forensic decision
B. Found, J. Ganas, The management of domain irrelevant context information in
MA
[28]
NU
making, J. Forensic Sci. 60 (2015) 1111–1112. doi:10.1111/1556-4029.12805.
forensic handwriting examination casework, Sci. Justice. 53 (2013) 154–158.
[29]
ED
doi:10.1016/j.scijus.2012.10.004.
E.J.A.T. Mattijssen, R.D. Stoel, W. Kerkhoff, Minimizing Contextual Bias in Forensic
EP T
Firearms Examinations, Wiley Encycl. Forensic Sci. (2015) 1–7. doi:10.1002/9780470061589.fsa1117. B.T. Ulery, R.A. Hicklin, J. Buscaglia, M.A. Roberts, Accuracy and reliability of
AC C
[30]
forensic latent fingerprint decisions, PNAS. 108 (2011) 7733–7738. doi:10.1073/pnas.1018707108. [31]
G. Langenburg, C. Champod, P. Wertheim, Testing for potential contextual bias effects during the verification stage of the ACE-V methodology when conducting fingerprint comparisons., J. Forensic Sci. 54 (2009) 571–82. doi:10.1111/j.15564029.2009.01025.x.
[32]
Office of the Inspector General, A Review of the FBI’s Handling of the Brandon
ACCEPTED MANUSCRIPT Mayfield Case, US Department of Justice, Office of the Inspector General, Oversight and Review Division, 2006. [33]
Office of the Inspector General, A review of the FBI’s progress in responding to the reccomendations in the Office of the Inspector General Report on the fingerprint
[34]
PT
misidentification in the Brandon Mayfield case., US Department of Justice, 2011. M.C. Taylor, N.K.P. Osborne, Letter to the Editor—A Contribution to Contextual
RI
Information Management in Bloodstain Pattern Analysis: Preliminary Idea for a Two-
SC
Step Method of Analysis, J. Forensic Sci. 63 (2018) 341. doi:10.1111/1556-
[35]
G. Whitman, R. Koppl, Rational bias in forensic science, Law, Probab. Risk. 9 (2010)
MA
69–90. doi:10.1525/sp.2007.54.1.23. [36]
NU
4029.13705.
M. Page, J. Taylor, M. Blenkin, Context effects and observer bias--implications for
N.K.P. Osborne, R. Zajac, An imperfect match? Crime-related context influences fingerprint decisions, Appl. Cogn. Psychol. 30 (2016) 126–134. doi:10.1002/acp.3180.
AC C
[37]
EP T
4029.2011.01903.x.
ED
forensic odontology., J. Forensic Sci. 57 (2012) 108–12. doi:10.1111/j.1556-
ACCEPTED MANUSCRIPT Highlights
EP T
ED
MA
NU
SC
RI
PT
The development and trial of a blind peer-review procedure in laboratory-based BPA Item re-assessed by examiner blind to case circumstances and original opinion Simple and practical way to manage contextual information as a way to reduce bias Example for other agencies who wish to adopt context management procedures Suitable for other disciplines; e.g. shoeprint, fabric damage, physical fits
AC C
management: An example of the review of laboratory-based
Nikola K.P. Osborne, Michael C. Taylor PII: DOI: Reference:
S1355-0306(18)30006-6 doi:10.1016/j.scijus.2018.01.001 SCIJUS 715
To appear in:
Science & Justice
Received date: Revised date: Accepted date:
5 June 2017 9 January 2018 12 January 2018
Please cite this article as: Nikola K.P. Osborne, Michael C. Taylor , Contextual information management: An example of independent-checking in the review of laboratory-based bloodstain pattern analysis. The address for the corresponding author was captured as affiliation for all authors. Please check if appropriate. Scijus(2018), doi:10.1016/ j.scijus.2018.01.001
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
ACCEPTED MANUSCRIPT Contextual Information Management: An example of independent-checking in the review of laboratory-based Bloodstain Pattern Analysis Paper prepared for submission to Science & Justice Nikola KP Osbornea*1 & Michael C Taylorb a
PT
The Institute of Environmental Science & Research (ESR), PO Box 29181, Christchurch 8540, New Zealand; [email protected] b
SC
RI
The Institute of Environmental Science & Research (ESR), PO Box 29181, Christchurch 8540, New Zealand; [email protected]
AC C
EP T
ED
MA
NU
*Corresponding author.
1
Please note, Dr. Osborne is now based at the University of California Irvine, Department of Criminology, Law, and Society, 2340 Social Ecology II, Irvine, CA 92697-7080
ACCEPTED MANUSCRIPT Abstract This article describes a New Zealand forensic agency’s contextual information management protocol for bloodstain pattern evidence examined in the laboratory. In an effort to create a protocol that would have minimal impact on current work-flow, while still effectively removing task-irrelevant contextual information, the protocol was designed following an in-
PT
depth consultation with management and forensic staff. The resulting design was for a
RI
protocol of independent-checking (i.e. blind peer-review) where the checker’s interpretation
SC
of the evidence is conducted in the absence of case information and the original examiner’s notes or interpretation(s). At the conclusion of a ten-case trial period, there was widespread
NU
agreement that the protocol had minimal impact on the number of people required, the cost, or the time to complete an item examination. The agency is now looking to adopt the protocol
MA
into standard operating procedures and in some cases the protocol has been extended to cover other laboratory-based examinations (e.g. fabric damage, shoeprint examination, and physical
ED
fits). The protocol developed during this trial provides a useful example for agencies seeking
EP T
to adopt contextual information management into their workflow.
AC C
Keywords: Forensic science, bloodstain pattern analysis, cognitive bias, blind peer-review, independent check, task-relevant information.
ACCEPTED MANUSCRIPT 1. Introduction The past decade has seen an increase in discussion [e.g. ,1–13] and research [e.g. ,14–25] relating to the issue of contextual bias in forensic decision-making. The main concern highlighted by commentators is that information irrelevant to a forensic examiner’s task may unconsciously influence (or bias) the examiner’s opinion. Indeed, the recent US President’s
PT
Council of Advisors on Science and Technology (PCAST) report [11] released the following
RI
call: “because contextual bias may have an impact on experts’ decisions, courts should assess the measures taken to mitigate bias during casework” [11 p. 100]. Such statements make it
SC
increasingly difficult for forensic agencies and practitioners to simply ignore the potential for
NU
extraneous contextual information to have a biasing influence on their opinions. At the forefront of recommendations to address the issue of contextual bias is a requirement
MA
to have examiners adopt contextual information management (CIM) procedures [1,6,9,12,13,26]. The main objective of CIM is to shield examiners from potentially biasing
ED
information, and that which is not relevant to their task, while still allowing access to task-
EP T
relevant information. The main CIM approaches discussed in the literature include: 1) (linear) sequential unmasking [5,27], 2) the context-manager model [26,28,29], and 3) blind peer-
AC C
review [12,30].
Sequential unmasking [5] and linear sequential unmasking [27] are recommended for sourcelevel comparison disciplines (e.g., DNA, fingerprints, and shoeprint comparisons), and specify the optimal order in which to examine the forensic material. Forensic experts should examine the latent (i.e. crime-scene) material prior to being exposed to the reference (i.e. suspect/known) material. In this way, they are working from the evidence to the suspect. Linear sequential unmasking also requires examiners to state levels of confidence in their opinion regarding the material under examination [27]. This method allows for flexibility in the decision-making process, but it also requires that the examiner be transparent in how s/he
ACCEPTED MANUSCRIPT has incorporated contextual information into his/her analysis, and what influence this information has had. The context-manager model, is an alternative approach whereby a context-manager has access to all of the contextual information, but only passes on to the examiner that which is relevant for the examination. In this way, the examiner conducts his/her examination in the
PT
absence of potentially biasing information, yet someone (i.e. the context-manager) still has
RI
access to the contextual information in order to determine the type of examination that might
SC
be necessary, and to put the examination and subsequent results in context with the case as a whole. This approach has been reported in forensic document [28] and firearm [26,29]
NU
examinations.
MA
Peer-review—an assessment or review of an examiner’s opinion and case notes—is an opportunity for any errors to be “picked up” prior to the case report being finalised. While
ED
many laboratories routinely perform a technical2 or administrative3 review of cases, an independent reassessment (from here on in, referred to as independent-check) of each
EP T
individual item examined is not so commonplace [30]. Independent checks are when the checker does not have access to the case information or the
AC C
original examiner’s conclusions. If a checker is not shielded from the same potentially biasing case information, it is possible that s/he will experience the same bias as the original examiner. In addition, knowledge of the previous examiner’s conclusion may introduce an additional source of bias. In fingerprint interpretation, for example, the linear ACE-V (assessment, comparison, evaluation – verification) methodology has been criticised for creating an expectation that a latent and comparison print share the same source [31,32]. That 2
A technical review is the review of the case notes, report, and conclusions. This is performed by another scientist. 3 An administrative review is to check for typographical and other non-examination-related errors. This does not necessarily have to be completed by a scientist.
ACCEPTED MANUSCRIPT is because only opinions which indicate a same-source conclusion are reviewed. Some forensic agencies have now adopted a methodology whereby all opinions are reviewed (including those that suggest exclusion or are inconclusive), and this is done in way that means the reviewer is not aware of the original opinion [33]. Bloodstain Pattern Analysis (BPA) is the analysis of the size, shape, and distribution of
PT
bloodstains. The methods for applying CIM in BPA are less clear than in other pattern
RI
comparison or source-conclusion forensic disciplines. BPA generally requires analysts to
SC
determine a mechanistic cause of the pattern (i.e. activity level opinion), and to put the bloodstain patterns in context with other elements of the scene. In these cases, some
NU
contextual information might be highly relevant – even if it is potentially biasing.
MA
Indeed the provision of contextual information has been identified as a potential source of error in BPA, with two systematic studies [14,15] showing that negatively-biasing case
ED
information (i.e. information suggesting an incorrect pattern type) increased the rate of incorrect pattern classifications compared to neutral or positively biasing case information
EP T
(i.e. information suggesting the ground truth pattern type). These studies did not include a condition that resembled a peer-review process, and therefore it is not clear whether or not a
AC C
second examiner would have identified and corrected these errors. It is clear, however, that bloodstain pattern analysts should recognize and address the potential for contextual information to influence their opinions. A New Zealand forensic agency, the Institute of Environmental Science and Research (ESR)4 , is actively endeavouring to minimise the potential for bias in their forensic examinations. This paper describes the development and trial of a contextual information
4
ESR is an independent government agency that is the sole provider of forensic services to the New Zealand Police. The Forensic Business Group comprises three Service Centres, located in Auckland, Kenepuru, and Christchurch.
ACCEPTED MANUSCRIPT management protocol for BPA within this agency. We present feedback after a trial period of ten cases. 2. Method 2.1 Protocol Development The first step in developing a protocol was to understand current procedures within each of
PT
ESR’s three forensic laboratories. We interviewed a cross section of management and
RI
scientific staff about: 1) current procedures for the examination of bloodstains at a crime
SC
scene and for bloodstained items examined in the laboratory, 2) the relevance of various types of information in BPA, and 3) the feasibility and practicality of introducing a context
NU
management system into current procedures.
MA
All those interviewed recognised that bias could be an issue in forensic examinations, however, there were concerns that extreme context management (i.e. the removal of all case information) could have a negative impact on their ability to effectively conduct a scene
ED
examination. In particular, scientists identified the need for basic case information in order to
EP T
determine the purpose of an examination (e.g. to inform which resource kits to take to the scene and the types of examinations required to determine and answer the investigative questions). For items examined in the laboratory, however, interviewees reported that the
AC C
need for case information is largely limited to identifying the case in the Laboratory Information Management System (LIMS). Although case information may not be needed for the item examination per se, the case manager who attends the scene generally also conducts the laboratory examinations for that case. The case manager, therefore, has the case information as a result of the scene examination. Efforts to have items obtained from a scene examination sent to an alternative laboratory, where a scientist who has no knowledge of the case circumstances can perform the item examination, were deemed possible but impractical.
ACCEPTED MANUSCRIPT Based on the interviews and examination of work-flow, we determined that an independentcheck (sometimes referred to as a blind peer-review) was the most appropriate and practical context management system to adopt for laboratory-based item examinations in BPA at ESR. ESR routinely uses a review process which includes review of the case notes and report, and can also include re-examination of an item (i.e. a bench peer-check). Currently, this bench
PT
peer-check is not always independent (i.e. blinded). In the proposed methodology, items are
RI
still to be examined by the case manager; however, a non-scene-attending scientist from the
SC
same laboratory performs a separate, independent analysis without any of the potential bias that could result from conducting the scene examination or from knowledge of the case
NU
manager’s opinion(s). 2.2 Trial Procedure
MA
Before deciding whether or not to recommend independent-checking as a standard operating procedure, we conducted a trial to assess the impact of this CIM protocol. The trial was
ED
conducted over a twelve week period during which the CIM protocol was applied to ten cases
EP T
that were received across ESR’s three laboratories as part of their routine casework. For the trial period, cases included in the trial were those in which bloodstain pattern analysis
AC C
was required on items submitted to the laboratory. These could include items that came from a scene that ESR attended, or items that were submitted by police but ESR did not attend the scene. Cases not included in the trial were those where the items requiring examination were sampled for DNA analysis but the examiner did not conduct an interpretation of any bloodstaining. The steps for the independent check were: 2.3 Independent-Check Protocol 1. Case manager (CM) assigned to the case.
ACCEPTED MANUSCRIPT 2. CM assigns an independent-checker (IC). IC informed that information regarding this case should be avoided where possible.5 3. Senior technician (ST) or CM draws a diagram of the item. ST screens and samples for blood, identifying areas of staining and recording the results of any tests performed. This diagram is replicated before any bloodstain pattern interpretations are made. The original is passed to the CM and the copy is put aside for the IC (or vice versa6 ).
PT
4. CM carries out BPA on the item(s) as per standard procedures, thoroughly documents all interpretations, and provides evidence–based conclusions.
RI
5. CM’s interpretations are omitted from any material that is to be passed to the IC.
SC
6. All case information is also omitted from material to be passed on to the IC. 7. IC conducts BPA on the item(s). IC thoroughly documents all interpretations on the 7
NU
diagram provided, and provides evidence-based conclusions.
8. CM compares IC’s interpretations with his/her interpretations. 9. If any inconsistencies between CM’s and IC’s interpretations arise, the standard
MA
laboratory procedure to resolve inconsistencies is followed. 10. Technical review completed by IC or additional scientist after context-managed
2.4 Information Management
ED
interpretations are made.8
was available:
Results of tests for blood Areas of staining identified by the CM/ST Location of bloodstaining on the item and location of other bloodstains in the scene. Location of the item in the scene
AC C
EP T
During the trial, the independent-checker could have access to the following information, if it
The following information was not made available to the independent-checker: 5
IC was not always decided this early in the procedure. He or she might have been assigned later depending on availability of scientists at the time. As a matter of course, however, only case managers and other sceneattending staff are aware of the case circumstances. 6 Because the IC does not have access to the CM’s notes and interpretations but only the diagram and the item itself, it is possible for the IC to conduct his/her examination first. The order of examination (i.e., whether CM or IC does his/her examination first) is not important; what is important is that the person conducting the second examination is not aware of the first examiner’s conclusions. 7 We did not stipulate a time frame for analysts to complete this, or any other step. Analysts were to proceed as they usually would, taking into account case load, urgency, and any other factors outside of our control. 8 If the IC is also the technical reviewer then by virtue of his/her role in determining if the case notes support the reported conclusions, he/she will be exposed to the contextual information and CM’s opinions in this step.
ACCEPTED MANUSCRIPT
CM’s interpretations Type of case Operation name Police briefing/Case circumstances Names of persons involved Alleged activities within the scene Police theory Witness statements Medical findings Other forensic reports
PT
In addition, informal discussion between peers about the case was deliberately limited. Upon
RI
completion of each case examination and independent check, case and item examination
SC
notes and interpretations were passed to the researchers. The trial was over once all 10 steps
NU
were completed on 10 cases that met the inclusion criteria. 3. Results
MA
As this was a trial conducted in the course of actual forensic casework, we could not assess examiners’ accuracy (i.e. we did not know the ground truth). We were able to observe,
ED
however, whether or not the independent-check process resulted in any disagreements or inconsistencies between the case manager’s and independent-checker’s opinions, and assess
EP T
if and how these were resolved. In nine out of the ten cases, there were no inconsistencies between the case manager’s and the independent checker’s interpretations. In one of the ten
AC C
cases, a pattern type identified as possible by the case manager, was not included (although not excluded) as a possible pattern type by the checker. This case and how the inconsistency was resolved is described below. 3.1 Case Example 3.1.1. Contextual information In this case, the case manager attended the scene and was aware of the following information:
Male A had seen a photo of his ex-partner with another man and became upset.
Male A arrived at ex partner’s address with a kitchen knife concealed in his pocket.
Male A punched the glass ranch slider, smashing it, cutting his wrist.
ACCEPTED MANUSCRIPT
Male A grabbed ex-partner and another occupant, pushing them into the living room.
Male A kicked and pushed occupants.
Three occupants wrestled Male A to the ground before he could use the knife on them.
3.1.2. Items submitted for examination One pair of trouser pants recovered from Male A was submitted to the laboratory for
PT
examination for blood and bloodstain pattern analysis. 3.1.3. Purpose of examination
RI
The stated purpose of this examination was to determine the cause of any bloodstaining on
SC
Male A’s trouser pants.
NU
3.1.4. Interpretation results
The following is a summary of the case manager’s interpretation notes (bloodstain pattern
MA
classification underlined):
“Airborne blood generated from force/impact or from blood into blood near lower R
ED
leg”
“Nothing in these stains to indicate cough/sneeze (expirated) is likely”
EP T
The following is a summary of the independent checker’s interpretation notes (bloodstain pattern classification underlined):
“Collection of fine spots (< 1mm) on lower shin/ankle at front of right leg = Close
AC C
proximity to impact of exposed source of blood” 3.1.4 Resolution
In this example, the case manager had indicated blood dripping into blood and impact as possible mechanisms for the blood deposition on the lower right leg of the trousers. The independent-checker had only included impact in his/her assessment. S/he neither included nor excluded blood dripping into blood as a possible account for the pattern. While these differing pattern mechanism accounts are not a disagreement per se, the inconsistency warranted further investigation. A second independent-check from an additional scientist also
ACCEPTED MANUSCRIPT resulted in an opinion that the pattern could be the result of impact or blood dripping into blood. Furthermore, several experiments were conducted in an attempt to replicate the patterns observed on the trousers. Using mechanisms that could create both blood dripping into blood and impact patterns, the case manager and independent-checker determined that either of these pattern mechanisms could account for the observed pattern on the trousers.
PT
The results of the experimentation and third examination meant that both impact and blood
RI
dripping into blood were included as mechanisms in the final report. The case manager
SC
reflected that the knowledge of Male A’s injury (cut wrist) meant that s/he considered whether the blood could have dripped onto blood present on the floor (or another surface
NU
close to the cuff of the trousers), and resulted in satellite spatter from blood dripping into
3.2 Summary of remaining cases
MA
blood.
The remaining cases included items of bloodstained clothing, shoes, and wood. Across these
ED
cases, analysts identified various pattern classification types including transfer, spatter, and
EP T
passive staining.
3.3 Analysts’ response to the trial
Overall, those involved in the trial responded positively to the introduction of an
AC C
independent-check system. Feedback from the scientists and technicians involved in the first ten cases included:
“[The method] is minimal extra hassle and time, [we] will continue the process as standard practice from now on.” - Senior Scientist “[The method] takes slightly longer as the reviewer has to write their own notes from scratch. However, it ensures that a true blind peer-check occurs.” – Senior Scientist “We will continue to use it, and should use it for other evidence types that require bench review (i.e. fabric damage, footwear comparisons, and physical fits).” – Senior Scientist
ACCEPTED MANUSCRIPT “At the technician level, the only added task is to make an extra copy of the diagram. My opinion is that it should continue onwards.” – Senior Technician 4. Discussion This paper describes the trial of a methodology of contextual information management in laboratory-based bloodstain pattern analysis, where a second assessment of the evidence
PT
occurred in the absence of any contextual case information or the original examiner’s notes
RI
and opinion(s). This trial was conducted to assess the feasibility of an independent-check protocol in practice. There were no reports of adverse consequences of the protocol, and the
SC
examiners who participated reported that they appreciated what it was trying to achieve and
NU
that it gave them more confidence that their opinions were unbiased. Introducing the protocol as standard practice, therefore, appears to be both achievable and welcomed.
MA
In nine out of ten cases, both the case manager and independent-checker reached the same opinion. In the one case where the case manager and independent-checker proffered
ED
inconsistent opinions; the case manager, by his/her own admission, was influenced by the
EP T
case circumstances and indicated blood dripping into blood as a possible mechanism because of these circumstances, not just because of the pattern characteristics. While this could be construed as bias, we could also contend that the contextual information has in fact assisted
AC C
the case manager to form a relevant hypothesis. The value of the independent-check here, therefore, is that it has identified a potentially biased opinion and has forced steps to reconsider the opinion in an unbiased way. The fact that this reconsideration—second independent-check and experimentation—supported the original (possibly biased) opinion in no way justifies the automatic inclusion of contextual information in BPA; rather, it points to the need to carefully manage its introduction. Furthermore, aside from raising questions about the reliability of the first independentchecker’s opinion (i.e. if the observations alone supported blood dripping into blood, then the
ACCEPTED MANUSCRIPT checker should have included this pattern mechanism), this inconsistency highlights at least two important issues for consideration in the management of contextual information in BPA, and for understanding BPA methodology in general. First, it appears that there may be a disadvantage to relying solely on blind examinations (i.e. where no examiner has access to contextual information at any point). Because contextual
PT
information may help an examiner to consider possible hypotheses to test, an absence of this
RI
information could result in a failure of abductive reasoning. Conversely, however, if all
SC
examinations of the same evidence are conducted with contextual information (i.e. check not independent), then any undue influence of the contextual information might never be
NU
recognized. An independent-check, therefore, helps to strike the delicate balance between
MA
having too much information and too little information.
Second, the inconsistency gives us a deeper understanding of the judgemental process in BPA. In this example, the case manager has used the contextual information to render a
ED
reconstructive theory that the blood could have dripped from the trouser-wearer’s injured
EP T
hand and onto the floor and then back onto the cuff of the trousers, thus creating a pattern resulting from blood dripping into blood. The reconstructive theory, therefore, has informed
AC C
the classification (abductive reasoning), rather than the classification informing the reconstructive theory. It is clear that BPA involves complex reasoning processes, and while contextual information may be biasing, it may also have a part to play in the development of relevant hypotheses to test. Here, we recognize that current bloodstain pattern classification methods and terminology do not offer a clear distinction between opinions related to an analyst’s observations (i.e. pattern characterization) and those related to his/her interpretations (i.e. mechanism determination, scene reconstruction). As such, understanding the relevance of contextual information, and how to manage it for each task, is not a simple endeavour in BPA. Until bloodstain pattern
ACCEPTED MANUSCRIPT classification methods and terminology clearly separate the currently overlapping components of observation and interpretation [34], we offer that an independent-check can help to disentangle the relative contribution of contextual information versus the contribution of the observations to the examiner’s opinion. This disentanglement is of great importance if the relevance or accuracy of the contextual information upon which an examination has relied
PT
upon is contested, or if the observations alone do not support the proffered opinion.
RI
As this is a relatively new methodology, we may need to consider making some changes and
SC
adjustments as difficult or complex cases arise. Complex cases might require discussion between two examiners prior to a conclusion being reported. Because complexity and
NU
ambiguity increases the potential for extraneous information to influence decision-making [1,12,35–37], complex examinations are most at risk of resulting in biased conclusions. In
MA
these cases, the case manager could consider sending the item to another examiner outside of his/her laboratory. Furthermore, examiners are likely to profit from engaging one another in
ED
discussions regarding how they have addressed context management for difficult or unusual
EP T
cases.
Although the procedure was originally designed for the examination of bloodstained items, it
AC C
has the potential to benefit all forensic disciplines. Indeed, scientists in this trial reported that they began to include other evidence types (e.g. shoeprint comparison, fabric damage and physical fits) even though they were not prompted to do so for the purpose of our trial. An additional benefit of this procedure is that it could allow for blind testing of examiner proficiency. That is, items for which the ground-truth is known, could be presented to the checkers as though they were case items. Testing in this way provides a more realistic assessment of proficiency than tests in which the examiner knows that he or she is being tested. The PCAST report [11] acknowledges the difficulties associated with implementing
ACCEPTED MANUSCRIPT blind proficiency testing, however, it recognizes the advantages and expects that all forensic laboratories include blind-testing into their routine work-flow within the next five years. Although we have presented a practical methodology for managing contextual information in the laboratory, we have not yet addressed the more difficult task of managing extraneous information at a crime scene. Scene-going forensic examiners cannot simply operate in a
PT
vacuum; the (potential) circumstances of the crime are inherent in the scene, after all that is
RI
what the examiner is there to assess. Furthermore, analysts are usually under (both implicit
SC
and explicit) pressure to examine the scene quickly and efficiently. For many forensic agencies, it is neither practical nor efficient to have two analysts perform independent scene
NU
examinations. Although checkers can conduct their BPA review through photographs and diagrams, these do not provide the same spatial representation that a scene does. The ability
MA
to capture a 3D representation of the scene may go some way in providing the opportunity for a second analyst to “revisit” the scene blind to contextual information and the original
ED
examiner’s opinions.
EP T
There are a number of additional points worth making in regards to the implementation of contextual information management for bloodstain pattern analysis: Contextual information regarding high profile cases may be difficult to avoid,
AC C
particularly in places where homicide rates are relatively low and case information is widely publicised.
Bloodstain pattern evidence, by its very nature, creates an expectation or insinuation that violence or pain has occurred. Furthermore, it is often only the ‘serious’ cases that forensic services are requested for. It is unclear to what extent this inherent and unavoidable contextual information could play a role in analysts’ decision-making.
At times there may only be a small number of analysts with an agency who are able to perform a BPA examination. If all available analysts have attended the scene (and therefore are aware of the case circumstances) it might be appropriate to send items to another laboratory to be reviewed.
ACCEPTED MANUSCRIPT
In laboratories where scientists often perform examinations across multiple forensic disciplines, there is a chance that checkers will have worked on other aspects of the case and therefore know the case circumstances. Ideally, items should be sent to an alternative laboratory to be checked by a scientist blind to the case circumstances, but at a minimum the check should always be performed blind to the original examiner’s interpretation, and examiners should avoid discussing the results of other forensic
PT
examinations. 4.1 Conclusions
RI
We have developed and trialled a simple and practical contextual information management
SC
procedure for laboratory-based bloodstain pattern analysis. In nine of the ten trial cases, both the case manager and independent checker reached the same opinion. In the single case in
NU
which there was a difference of opinion, this was resolved by enlisting a second independent
MA
checker and through experimentation.
Adherence to the described procedure will decrease the potential for extraneous contextual
ED
information to influence the independent checker’s opinion. Furthermore, assessing any difference between the case manager’s and the independent checker’s opinion allows for
EP T
some understanding of the relative contribution of the contextual information to the contextually- informed opinion. This understanding should ultimately lead to better-informed
AC C
and considered determinations.
This procedure also allows the courts to assess that the agency has taken steps to shield their analysts from potentially biasing information. Given the results of this trial and the positive response from participants to the independent-check protocol, the acceptance of such a protocol into standard operating procedures appears to be both achievable and worthwhile. The protocol developed during this trial provides a useful example for agencies seeking to adopt contextual information management into their workflow. Acknowledgements
ACCEPTED MANUSCRIPT This project was funded by a grant from the National Institute of Justice (NIJ #2014-DN-BXK032). The authors gratefully acknowledge the forensic staff and management at The
AC C
EP T
ED
MA
NU
SC
RI
PT
Institute of Environmental Science and Research (ESR) for their participation in the trial.
ACCEPTED MANUSCRIPT
5. References [1]
S.M. Kassin, I.E. Dror, J. Kukucka, The forensic confirmation bias: Problems, perspectives, and proposed solutions, J. Appl. Res. Mem. Cogn. 2 (2013) 42–52.
[2]
PT
doi:10.1016/j.jarmac.2013.01.001. I.E. Dror, The ambition to be scientific: Human expert performance and objectivity,
I.E. Dror, Cognitive forensics and experimental research about bias in forensic
SC
[3]
RI
Sci. Justice. 53 (2013) 81–82. doi:10.1016/j.scijus.2013.03.002.
[4]
NU
casework, Sci. Justice. 52 (2012) 128–130. doi:10.1016/j.scijus.2012.03.006. R. Zajac, N.K.P. Osborne, L. Singley, M.C. Taylor, Contextual Bias: What Bloodstain
[5]
MA
Pattern Analysts Need to Know, J. Bloodstain Pattern Anal. 31 (2005). D.E. Krane, S. Ford, J.R. Gilder, K. Inman, A. Jamieson, R. Koppl, I.L. Kornfield,
ED
M.D. Risinger, N. Rudin, M.S. Taylor, W.C. Thompson, Sequential unmasking: A
EP T
means of minimizing observer effects in forensic DNA interpretation, J. Forensic Sci. 53 (2008) 1006–1007. doi:10.1111/j.1556-4029.2008.00787.x. G. Edmond, A. Towler, B. Growns, G. Ribeiro, B. Found, D. White, K. Ballantyne,
AC C
[6]
R.A. Searston, M.B. Thompson, J.M. Tangen, R.I. Kemp, K. Matire, Thinking forensics: Cognitive science for forensic practitioners, Sci. Justice. in press (2016) 1– 27. doi:http://dx.doi.org/10.1016/j.scijus.2016.11.005. [7]
R.D. Stoel, I.E. Dror, L.S. Miller, Bias among forensic document examiners: Still a need for procedural changes, Aust. J. Forensic Sci. 46 (2014) 91–97. doi:10.1080/00450618.2013.797026.
[8]
B. Found, Deciphering the human condition: the rise of cognitive forensics, Aust. J.
ACCEPTED MANUSCRIPT Forensic Sci. 47 (2015) 386–401. doi:10.1080/00450618.2014.965204. [9]
S.A. Cole, Implementing counter-measures against confirmation bias in forensic science, J. Appl. Res. Mem. Cogn. 2 (2013) 61–62. doi:10.1016/j.jarmac.2013.01.011.
[10]
National Research Council, Strengthening Forensic Science in the United States: A
[11]
PT
Path Forward, The National Academies Press, Washington, D.C., 2009. President’s Council of Advisors on Science and Technology, Forensic Science in
M.D. Risinger, M.J. Saks, W.C. Thompson, R. Rosenthal, The Daubert/Kumho
SC
[12]
RI
Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods, 2016.
NU
Implications of Expectation and Suggestion Observer Effects in Forensic Science: Hidden Effects of Expectation and Suggestion, Calif. Law Rev. 90 (2002) 1–56. W.C. Thompson, What role should investigative facts play in the evaluation of
MA
[13]
scientific evidence?, Aust. J. Forensic Sci. 43 (2011) 123–134.
M.C. Taylor, T.L. Laber, P.E. Kish, G. Owens, N.K.P. Osborne, The Reliability of
EP T
[14]
ED
doi:10.1080/00450618.2010.541499.
Pattern Classification in Bloodstain Pattern Analysis, Part 1: Bloodstain Patterns on Rigid Non-absorbent Surfaces, J. Forensic Sci. 61 (2016) 922–927. doi:10.1111/1556-
[15]
AC C
4029.13091.
M.C. Taylor, T.L. Laber, P.E. Kish, G. Owens, N.K.P. Osborne, The Reliability of Pattern Classification in Bloodstain Pattern Analysis—Part 2: Bloodstain Patterns on Fabric Surfaces, J. Forensic Sci. 61 (2016) 1461–1466. doi:10.1111/1556-4029.13191.
[16]
I.E. Dror, D. Charlton, A.E. Péron, Contextual information renders experts vulnerable to making erroneous identifications, Forensic Sci. Int. 156 (2006) 74–78. doi:10.1016/j.forsciint.2005.10.017.
ACCEPTED MANUSCRIPT [17]
S. Nakhaeizadeh, I.E. Dror, R.M. Morgan, Cognitive bias in forensic anthropology: Visual assessment of skeletal remains is susceptible to confirmation bias, Sci. Justice. 54 (2014) 208–214. doi:10.1016/j.scijus.2013.11.003.
[18]
I.E. Dror, D. Charlton, Why experts make errors, J. Forensic Identif. 56 (2006) 600–
[19]
PT
616. J. Kukucka, S.M. Kassin, Do confessions taint perceptions of handwriting evidence?
RI
An empirical test of the forensic confirmation bias, Law Hum. Behav. 38 (2014) 256–
N.K.P. Osborne, M.C. Taylor, R. Zajac, Exploring the role of contextual information
NU
[20]
SC
270. doi:10.1037/lhb0000066.
in bloodstain pattern analysis: A qualitative approach, Forensic Sci. Int. 260 (2016) 1–
[21]
MA
8. doi:10.1016/j.forsciint.2015.12.039.
N.K.P. Osborne, M.C. Taylor, M. Healey, R. Zajac, Bloodstain pattern classification:
ED
Accuracy, effect of contextual information and the role of analyst characteristics, Sci.
[22]
EP T
Justice. 56 (2016) 123–128. doi:10.1016/j.scijus.2015.12.005. N.K.P. Osborne, S. Woods, J. Kieser, R. Zajac, Does contextual information bias bitemark comparisons?, Sci. Justice. 54 (2014) 267–273.
[23]
AC C
doi:10.1016/j.scijus.2013.12.005. J.H. Kerstholt, A. Eikelboom, T. Dijkman, R. Stoel, R. Hermsen, B. van Leuven, Does suggestive information cause a confirmation bias in bullet comparisons?, Forensic Sci. Int. 198 (2010) 138–142. doi:10.1016/j.forsciint.2010.02.007. [24]
J.H. Kerstholt, R. Paashuis, M. Sjerps, Shoe print examinations: Effects of expectation, complexity and experience, Forensic Sci. Int. 165 (2007) 30–34. doi:10.1016/j.forsciint.2006.02.039.
[25]
L.J. Hall, E. Player, Will the introduction of an emotional context affect fingerprint
ACCEPTED MANUSCRIPT analysis and decision-making?, Forensic Sci. Int. 181 (2008) 36–39. doi:10.1016/j.forsciint.2008.08.008. [26]
E.J.A.T. Mattijssen, W. Kerkhoff, C.E.H. Berger, I.E. Dror, R.D. Stoel, Implementing context information management in forensic casework: Minimizing contextual bias in firearms examination, Sci. Justice. 56 (2016) 113–122.
I.E. Dror, W.C. Thompson, C.A. Meissner, I. Kornfield, D.E. Krane, M.J. Saks, M.
RI
[27]
PT
doi:10.1016/j.scijus.2015.11.004.
SC
Risinger, Letter to the Editor- Context Management Toolbox: A Linear Sequential Unmasking (LSU) approach for mimimizing cognitive bias in forensic decision
B. Found, J. Ganas, The management of domain irrelevant context information in
MA
[28]
NU
making, J. Forensic Sci. 60 (2015) 1111–1112. doi:10.1111/1556-4029.12805.
forensic handwriting examination casework, Sci. Justice. 53 (2013) 154–158.
[29]
ED
doi:10.1016/j.scijus.2012.10.004.
E.J.A.T. Mattijssen, R.D. Stoel, W. Kerkhoff, Minimizing Contextual Bias in Forensic
EP T
Firearms Examinations, Wiley Encycl. Forensic Sci. (2015) 1–7. doi:10.1002/9780470061589.fsa1117. B.T. Ulery, R.A. Hicklin, J. Buscaglia, M.A. Roberts, Accuracy and reliability of
AC C
[30]
forensic latent fingerprint decisions, PNAS. 108 (2011) 7733–7738. doi:10.1073/pnas.1018707108. [31]
G. Langenburg, C. Champod, P. Wertheim, Testing for potential contextual bias effects during the verification stage of the ACE-V methodology when conducting fingerprint comparisons., J. Forensic Sci. 54 (2009) 571–82. doi:10.1111/j.15564029.2009.01025.x.
[32]
Office of the Inspector General, A Review of the FBI’s Handling of the Brandon
ACCEPTED MANUSCRIPT Mayfield Case, US Department of Justice, Office of the Inspector General, Oversight and Review Division, 2006. [33]
Office of the Inspector General, A review of the FBI’s progress in responding to the reccomendations in the Office of the Inspector General Report on the fingerprint
[34]
PT
misidentification in the Brandon Mayfield case., US Department of Justice, 2011. M.C. Taylor, N.K.P. Osborne, Letter to the Editor—A Contribution to Contextual
RI
Information Management in Bloodstain Pattern Analysis: Preliminary Idea for a Two-
SC
Step Method of Analysis, J. Forensic Sci. 63 (2018) 341. doi:10.1111/1556-
[35]
G. Whitman, R. Koppl, Rational bias in forensic science, Law, Probab. Risk. 9 (2010)
MA
69–90. doi:10.1525/sp.2007.54.1.23. [36]
NU
4029.13705.
M. Page, J. Taylor, M. Blenkin, Context effects and observer bias--implications for
N.K.P. Osborne, R. Zajac, An imperfect match? Crime-related context influences fingerprint decisions, Appl. Cogn. Psychol. 30 (2016) 126–134. doi:10.1002/acp.3180.
AC C
[37]
EP T
4029.2011.01903.x.
ED
forensic odontology., J. Forensic Sci. 57 (2012) 108–12. doi:10.1111/j.1556-
ACCEPTED MANUSCRIPT Highlights
EP T
ED
MA
NU
SC
RI
PT
The development and trial of a blind peer-review procedure in laboratory-based BPA Item re-assessed by examiner blind to case circumstances and original opinion Simple and practical way to manage contextual information as a way to reduce bias Example for other agencies who wish to adopt context management procedures Suitable for other disciplines; e.g. shoeprint, fabric damage, physical fits
AC C