Digital Investigation (2005) 2, 71e73
www.elsevier.com/locate/diin
EDITORIAL
Growing pains
As interest in a particular field grows, new demands are placed on the practitioners that can compromise the fundamental principles and goals of their work. A vivid historical example of this trend may be seen in the development of scientific archaeology in 1930s Italy. After several decades of careful archaeological excavation of ancient sites such as Pompeii and Ostia, Mussolini’s government realised their political potential (Gessert, 2001). In the case of Ostia, pressures such as the need for government funding caused the director to abandon his earlier meticulous excavations and conduct a hurried dig using heavy machinery, uncovering large areas in a short time. Although a significant amount of material was exposed, the speed of this process caused the obliteration of valuable evidence, disturbed the context of artifacts, and made the thorough documentation of findings impossible. Ultimately, archaeologists were compelled to compromise their principles and harm their sites in order to get funding for their work. The resulting interpretations of items uncovered during this period are necessarily limited and can be called into question because they are founded on weak evidence. In many respects, the field of digital forensics is going through a similar growth phase, with increasing demands being placed on digital investigators to process large volumes of data and produce probative results quickly. Like the excavations at 1742-2876/$ - see front matter ª 2005 Published by Elsevier Ltd. doi:10.1016/j.diin.2005.04.004
Ostia, haste in processing digital evidence can compromise an investigation, particularly when unrefined tools are used. Digital investigators frequently rely on new tools and techniques applied to novel situations, and experience problems during active investigations caused by bugs in forensic software. It can take knowledgeable, experienced individuals a significant amount of time to address such problems and ensure that solid digital evidence is produced. As important as it is to raise awareness about the impressive applications of digital forensics, it is equally important to refine core capabilities and publicise the current limitations and challenges. Decision-makers require full information to allocate sufficient resources and time for tool testing and troubleshooting, improving practices, and training. Unless we inform decision-makers about problems that we regularly encounter during digital investigations, we may find ourselves compelled to perform our work hastily at the expense of fundamental principles of our field, such as accuracy and completeness. Furthermore, we may be penalized for failing to meet unrealistic demands.
Successes and failures It is natural that exciting investigative successes receive more attention than relatively mundane failures. The use of digital evidence to apprehend dangerous criminals receives major press, whereas missed investigative opportunities due to mistakes and tool failures are regularly swept under the carpet. Advanced techniques such as the recovery of partially overwritten data using equipment for testing hard drives, called ‘‘spin stand testers,’’ have received significant attention in the press (Knight, 2004; Khamsi, 2005) despite the fact that this recovery method has significant limitations, is
72 prohibitively expensive for most investigations, and has not been published or vetted in the scientific community. This over-representation of the spectacular is skewing the expectations of managers, attorneys, judges, and other decisionmakers. As a result, unrealistic demands are being placed on digital investigators, and these decisionmakers are surprised when investigations take longer and cost more than anticipated, or the evidence resulting from rushed processing contains errors. Because the shortcomings in tools and challenges in our field receive less publicity, many decision-makers are unaware of how difficult it can be to obtain and process large amounts of data from an organization’s computer systems in a forensically sound manner. Backup tapes provide a prime example of the disconnect between the reality and unrealistic expectations given the state of the field. Digital investigators are commonly asked to search backup tapes for relevant data under the assumption that it is a relatively straightforward task. In fact, processing magnetic tapes as evidence can be a mammoth undertaking, even when they are in good physical condition. As demonstrated by Paul Sanderson’s letter to the editor in this issue, knowledge of forensic preservation and examination of magnetic tapes is still developing. There are many different kinds of tapes that require specialized readers e even tapes made by the same vendor can require different tape readers. Tapes in large backup systems such as Tivoli Storage Manager only contain compressed fragments of data, making it effectively impossible to recover files after references to them are erased from the system’s index. Furthermore, extracting and converting certain types of data such as e-mail from backup tapes can be challenging and error prone. Even companies that specialize in this area may have trouble in extracting and converting data on backup tapes. Backup tapes are particularly problematic because it is hard to manage the thousands of tapes that are created in many organizations each year, and it is even harder to process large numbers of tapes under time pressure during an investigation. Organizations are being penalized due to problems with the management of large quantities of backup tapes. For instance, five financial institutions were fined a total of $8,250,000, in part, for relying on backup tapes for their required retention of e-mail data, when the tapes were overwritten, recycled, discarded or the e-mail data were unorganized and generally inaccessible (SEC, 2002).
Editorial By raising awareness of the difficulties in dealing with backup tapes and other sources of digital evidence, we can help managers prepare their organizations for litigation, and help attorneys, regulators and courts draft motions and production schedules that allow sufficient time and resources to perform the requested task.
Tool vendors Some forensic software vendors are placing added strain on digital investigators by promoting the strengths of their tools while unduly downplaying their weaknesses. Conversations among digital investigators are replete with horror stories of tool failures discovered during active investigations, resulting in lost evidence and incorrect information. For instance, tools for processing e-mail in digital investigations must be monitored closely to ensure accuracy. Critical shortcomings may include incomplete acquisition, failure to recover deleted messages, inaccurate removal of duplicate messages, and alteration of associated metadata. When bugs are reported to the software developers, they often recommend upgrading to the latest version without further explanation. The lack of formal bug-tracking systems prevents investigators from determining what problems exist and have been fixed in each version of the tools we use. It is not even clear whether the developers themselves understand the problems. Permitting vendors to downplay significant problems puts digital investigations in jeopardy and hinders improvements in the field. To compensate for the lack of robust quality assurance and bug tracking mechanisms, it is necessary for digital investigators to share their knowledge and evaluations of forensic tools. Although efforts such as the NIST Computer Forensic Tools Testing program (www.cftt.nist.gov), the Defense Cyber Crime Institute (www.dcfl.gov), and the tool review articles in this journal are making progress in this area, more published evaluations and tests are needed. Organizations should encourage their members to share knowledge of tool strengths and shortcomings, and other aspects of digital investigation through publication. In addition to motivating vendors to improve their tools, such contributions help decision-makers and digital investigators avoid dangerous pitfalls, and thereby increase the reputation of the examiners and their employers.
Conclusions As we strive to improve our tools and techniques, it is important to be open about current limitations in
Editorial digital forensics. If we do not publicise investigative challenges and software bugs, we risk harming ourselves, our clients, and ultimately the field.
References Gessert G. Urban spaces, public decoration, and civil identity in ancient Ostia, PhD Dissertation, Yale University; 2001. Khamsi R. Dusting for digital fingerprints. The Economist Technology Quarterly, March 12 2005; US ed. Knight W. Chasing the elusive shadows of e-crime, New Scientist, May 2004. Securities and Exchange Commission. Order instituting proceedings pursuant to Section 15(b)(4) and Section 21c of the
73 Securities Exchange Act of 1934, making findings and imposing cease-and-desist orders, penalties, and other relief. Deutsche Bank Securities, Inc., Goldman, Sachs & Co., Morgan Stanley & Co. Incorporated, Salomon Smith Barney Inc., and U.S. Bancorp Piper Jaffray Inc., Administrative Proceeding, File No. 3-10957; 2002. Available online at: http://www.sec.gov/litigation/admin/34-46937.htm.
Eoghan Casey, Editor-in-chief 1150 Connecticut Avenue, Suite 200, Washington, DC 20036, USA E-mail address:
[email protected]