Reporting
One of the least talked about and perhaps most important, and at the same time difficult aspects of digital analysis for examiners is communicating their results and findings clearly and concisely in a written report.
Keywords
Goals; case notes; report writing
Information in This Chapter
Introduction
Perhaps one of the most difficult aspects of digital analysis work is communicating results and findings. However, this is also perhaps the most important aspect of our jobs, as well. We write reports because we have to communicate our findings to someone else, be they a customer or an attorney, or another analyst, in such a manner that they can use that information to make critical business or legal decisions. As such, we need to clearly and concisely address the issues at hand and provide information on which the reader can base their decisions.
In this chapter, I will be sharing a method of reporting that has worked for me, as a consultant. This means that I’m usually hired by a company or organization to perform some sort of examination or assessment, and provide information describing my findings, usually in a written report (I have done work for customers who wanted verbal responses before they decided if they wanted something written). Most often, those reading my reports have been nontechnical executive or senior level management, and in some cases, attorneys. As such, I need to ensure that the information in my report can be understood and used by whoever is reading it to make the critical business or legal decisions with which they are faced.
While I have seen reports written by law enforcement analysts, I have not yet been in a position to do so myself. As such, I do not have a great deal of insight to offer for writing reports and communicating findings specifically for this segment of the digital forensics community. I have, however, used the approach discussed in this chapter from reporting findings of internal investigations, such as human resources (HR) and internal security incident investigations that remain within the organization and do not employ external third party or law enforcement assistance. The reporting methodology I have used as a consultant works equally well for these investigations. Hopefully some, if not most, of what I discuss in this chapter will be applicable to your needs.
Goals
Every examination or engagement starts with goals; after all, without some sort of goals, there would be no reason for the analysis, right? Someone wouldn’t want you to perform a considerable amount of analysis work and reporting unless they had a question that needed to be answered, right? Now, they may not have what you consider the “right” question, or they may have additional questions that only come about after you provide your findings, but my point is that every examination begins with a goal, or goals, of some sort.
While I was on active duty in the military, I heard a statement that stuck with me; “If you can’t communicate an issue on a 3×5 index card, you don’t know enough about it.” This is very true when it comes to defining the goals of an examination. Very often, I will hear analysts say that the customer could not define their goals, or the customer’s goals were very vague, such as “…find all bad stuff.” Most often, this is the result of minimal interaction with the customer. I once worked with an analyst who had returned from a customer’s site with the goals of “find all bad stuff.” They worked diligently on the case and sent in a report stating that they’d located not just a directory full of “hacking” tools, but also indications that the user in question had actually run these tools. The customer response was that the use of the “hacking” tools was part of that employee’s job—their job function was explicitly to test the security of the company systems and web sites, so the tools that were discovered were not “bad.”
Goals focus our attention and work. This is not to say that we focus so narrowly on the goals that we provide nothing more than a “yes” or “no” answer. However, goals do help us identify the analysis process and techniques that we would want to employ, what artifacts we need to be looking for or at, as well as what other information may be of use in order to help us answer the questions we have before us. For example, if a goal of an examination is to determine what information, if any, left the system then I would want to seek additional information regarding what led the customer to believe that information had left the system, what type of information (Office documents, etc.) might have left the system, and when they believe this might have happened. I might also want to know what other logs might be available from within the infrastructure, such as web proxy logs, email server logs. The point is that there are a lot of ways that information can make its way off of a system, and not all of them require the use of technological means to achieve this, so without narrowing down the goals of the examination, as well as the customer’s expectations, considerable time and money could be spent on an examination that would, in the end, have very little return on that investment.
Goals should be discrete, achievable, and manageable. We should be able to tell when we’ve met a goal, or if we don’t have enough information to adequately address a goal, and in such cases, we should be able to identify what else it is that we need to accomplish the goal. For example, I have done a good bit of dead box analysis; that is, analyzing images acquired from hard drives removed from systems after they’ve been shut down. In such cases, I have not had access to the system when it was in a live, running state; I have no information about processes that were running on the system, network connections initiated from the system, etc. So when a customer asks me to determine what information was exfiltrated off of the system several weeks (or even months) prior to when they contacted me, I know that this will be a difficult goal to achieve. After all, the only definitive way to determine what data was exfiltrated from a system is to capture and analyze traffic emanating from the system at that time; anything else might amount to varying degrees of speculation. Yes, it may be possible to locate a .rar archive of files in a staging folder, and the usual actions associated with the threat actor may be that they export these archives off of the system in some manner, but the fact is that unless you have a network traffic capture what you can parse and from which you can retrieve that archive, the exfiltration of the archive is speculation, at best.
Often, through the experience of multiple and repeated examinations, we may know what the customer is going to ask before they do. Companies that haven’t experienced a breach or intrusion, and don’t have experience with dealing with such incidents, will often start by asking if a system is infected with malware. This may seem to be a simple “yes” or “no” goal, but it’s really just the first in a series of questions that they’re going to be asking. “Was the system infected?” will usually lead to, “what were the capabilities of the malware?”, which will then lead to, “what data was exposed by the malware?” There is an inevitable refinement in the goals over time, as the analysis provides answers and those answers are processed. As such, it is a good idea to work with the customer up-front, and present these questions as possibilities, and let them know that you understand where they will be going with their questions. For consultants, this can have an impact on the work that you do, as if the customer keeps coming back with another question, you will have to go back and request additional hours in order to complete the work. Sitting with the customer ahead of time and working through the questions that they have, as well as those that they are likely to have, will allow you to provide a much more realistic estimate of how long they can expect the work to take. This has much more of an impact on the customer than just their wallet; many organizations are subject to some sort of compliance regulations, which in some cases will specify a time frame within which the customer needs to respond.
Incident triage
The goals of your examination will most likely originate through your initial contact with your “customer.” I use this term, because as analysts, we are not performing analysis for ourselves—we’re most usually performing analysis for someone else. We may do this while acting as a consultant, and the term “customer” is used in the classic sense, and synonymous with “client.” For a law enforcement analyst, the “customer” may be the prosecution team, or the courts. For an analyst working in a full-time employment position within an organization (i.e., part of the internal security team), the “customer” may be the HR director, the CIO, or someone to whom they are providing support and analysis. Regardless of the position in which you’re functioning, your analysis goals are going to come from that first contact with your “customer,” during those initial information gathering steps.
As such, it is very often helpful (and in some cases, critical) to have a checklist of questions handy for when you’re engaged with someone regarding an incident of any kind. As a consultant, I’ve used what can be referred to as a triage checklist of questions that will be asked of a customer, either when they first call, or when I get to meet with them. Often, it is also helpful to have the answers to these questions from the initial call in front of you when meeting with the customer, and then asking the questions again. In many cases, the customer may not have thought of the questions you asked, or when you’re sitting with them, there may be someone with more knowledge of the incident available. The questions asked will vary depending upon your experience, type of work that you do, who you work with, etc. For example, as a consultant who dealt with different customers from different verticals all the time—I might talk to a restaurant owner one week, and the next week meet with the CIO of a hospital—the questions I ask will likely be different from those asked by a law enforcement examiner.
One of the first things I ask a customer to do is provide a concise description of their understanding of the incident. This will often lead to either additional questions depending upon the nature or the description of the incident, and in many cases, has led to the customer agreeing with me that we need to have someone a bit more familiar with the incident on the call to answer questions. For example, I’ve been on the phone with someone who knew that there were 3000 server systems in the data center, but had no idea if any of them were involved in the incident. Keep in mind that when someone is calling for your assistance, it is likely that incident response or digital analysis is not what they do on a daily basis; if they’re calling about an incident that they’d like you to respond to, this may be the first time that they’ve encountered something like this. Ever. As such, having them describe the incident is a great way to get an understanding of what’s going on; for example, if they call and say, “we have a malware infection,” one of the first things I would like to know is, how do they know that? Getting them to describe the incident or issue is a great way to open up the conversation a bit and start getting not just a view into the issue, but what their concerns are, as well.
Another question I will ask is if there are any network diagrams and device logs available. It can be very helpful to understand which devices network traffic will have to traverse in order to make it from point A to point B, as well as which of those devices might be capable of logging. I’ve worked with organizations that have employed an automated process for archiving their firewall logs on a regular basis. Most web servers will maintain logs by default, but I have worked with organizations that had disabled logging on several of their web servers. If an incident was detected due to lateral movement within an organization (i.e., traffic did not leave the internal infrastructure and make it out on to the Internet), it can be very helpful to understand the devices the traffic would have traversed.
Case Notes
Maintaining case notes is extremely important to a case, especially when it comes to reporting. There is a well-known euphemism in the community about an analyst “getting hit by a bus,” which could refer to a wide range of events, such as that the analyst was transferred, on vacation, sick, that their computer ceased to function, or that the data somehow became corrupted. Without case notes, no one knows what that analyst was working on, what they had done, or what they had found. Often we look at the euphemism as just that—something that might happen, but to someone else. The need for case notes becomes very real when the euphemism becomes manifest, and we learn our lessons about keeping case notes the hard way, most often after the fact.
Another question to ask yourself when writing your case notes is, what happens when the customer comes back and asks questions six months or a year after you completed the original work and submitted your report? What’s funny—well, not really—about this is that it actually happens. I know many analysts have heard their manager say those words, and thought nothing of it. And then a year after they did the work, the customer did, in fact, come back and ask questions—questions that the analyst couldn’t answer because they didn’t keep case notes. And it doesn’t have to be a full year. Often, analysts may work several cases at the same time, as well as in rapid succession. Maintaining case notes allows you to differentiate your analysis, and even answer questions just a few weeks after you completed the work. As such, when discussing the need for case notes with another analyst, I will ask them, how will you be able to answer questions about a case six months or a year after you completed it? This may sound notional and far-fetched, but I’m sure that we can all recall having been on a conference call when we were asked questions about an examination had done, a full year after we had completed the work and sent the report to the customer. This sort of scenario can happen in a number of cases, such as with legal cases that don’t go to trial right away, as well as when an organization is fined or somehow penalized by a regulatory body and appeals that finding. Often, the reasons we give for keeping case notes seem contrived and unlikely to occur, but then we learn the value of having kept case notes when those reasons come true and actually do happen.
Another important benefit of maintaining case notes is that doing so reduces the risk of commingling case work in the analyst’s mind and memory. Even if you’re working just a few, or even two, cases without notes, you may often find yourself remembering details from one case while discussing another. I’ve seen this a number of times; I once spent a considerable amount of time discussing a case with another analyst, who would say things such as, “the time zone for the system was set for Egypt”; when I asked for clarification that the acquired system had been physically located in Baltimore, MD, the analyst corrected herself. She hadn’t been keeping case notes; as she was handling multiple cases, she was commingling the details from various cases. Doing this on the report, or in court, can have some significant repercussions.
Something else that analysts will often say about case notes is that they don’t keep everything; in particular, they don’t keep what they tried, but didn’t work. For example, some analysts will say that if they have a theory about something and they check a Registry value or perform a keyword search and they don’t find what they might have expected, they won’t document that aspect of their analysis. I say, why not? This can be very valuable information, supporting other findings, and helping you complete your analysis goals. Often, the absence of an artifact where you would expect to find one is itself an artifact. Not finding something can be as valuable as locating the data that you’re actually looking for. For example, during one particular case, I found that the user’s NTUSER.DAT Registry hive did not have a RecentDocs key; experience and testing showed me that when a new user profile is created on a system, the NTUSER.DAT hive has a RecentDocs key, albeit an empty one. I then used a tool to examine the hive file for deleted keys and values, and found the date and time of when the key had been deleted, and using that information was able to determine that the user had used a tool to purposely delete the key and its contents. Having this information documented in my case notes was invaluable in supporting my later findings.
So, how do you keep case notes? We’ve already talked about the need to keep case notes, but what application or tool should you use? Again, this is another question that I hear quite often. One of the tools I have used to great success is Forensic Case Notes from QCC Information Security (available online from http://qccis.com/resources/forensic-tools/casenotes-lite/). This tool was specifically designed to allow an analyst to keep case notes, and has a good deal of built-in functionality that can be extremely useful, such a hashing of entered data, the ability to add additional tabs, encryption, and language support. When I used this application, I found the rich text format of the text fields to be very useful for entering data, particularly if I wanted to include a picture in my notes.
If your analysts use multiple platforms, however, a Windows-only tool may not be the best choice. A number of applications support the document format used by MS Word (MS Office for Mac, OpenOffice for Linux, etc.), so this may be a more suitable choice of application platform for maintaining and sharing case notes. Documents in MS Word format can be created and accessed on a variety of platforms, including (but not limited to) an iPad. Case notes files can also be converted to PDF format for archival purposes, or for sharing with other analysts.
Documenting your analysis
I’m often asked, to what standard should case notes be kept? My response is simply this—if you “get hit by a bus,” someone should be able to take your case notes and replicate what you did, not just the steps you took, but your findings, as well. They should be able to follow your case notes like a recipe, and using the same data, same tools (and versions of the tools), and the same commands to launch those tools, be able to reproduce your findings. Based on your own training and experiences, as well as how well you document your case notes, the conclusions may be different, but another analyst should be able to run same version of the same tool against the same data and get the same results.
When keeping case notes and documenting your analysis, you should keep detailed information regarding the tools you use to parse and interact with the data, including the name and version of the tool. This can be very important, as the capabilities of tools tend to evolve over time. For example, when I started with the ISS Emergency Response Services (ERS) team, each analyst was provided a dongle and a copy of Guidance Software’s EnCase version 4.22. By the time I moved on from the team 3½ years later, EnCase was at version 6.17 or 6.19, and the capabilities of the tool had greatly expanded.
If you’re using free and/or open source tools, you should also document the location from where you retrieved a copy of the tool, in addition to the other information discussed thus far. Many times you may find a number of tools for parsing various types of data (files, data structures, etc.), all with the same name, but with vastly different capabilities and output formats. As such, knowing from where the tool was retrieved can be extremely important when it comes to answering questions at some point in the future. Also, documenting this information allows another analyst to review your work and easily understand what you had done to that point (remember the “hit by a bus” euphemism?).
Why does any of this matter? Remember that as stated in Chapter 1, tools provide a layer of abstraction over the data that analysts interface and interact with, often translating a binary stream or flag value into something readable by humans. An example of this is can be found in parsing the security accounts manager (SAM) Registry hive file—there is no actual string that says that an account is disabled; rather, it is a flag value, and when found, most tools will display some variation of “account disabled,” while the value itself is a single bit within a 32-bit section of a binary stream. Different tools written by different authors with different perspectives may parse or display things differently, and, as stated, tools will evolve over time. This is particularly true with open source tools. In 2012, I had found that the Windows Registry Recovery tool from MiTeC (found online at http://mitec.cz/wrr.html) did not correctly parse “big data,” or value data larger than about 3 kilobytes or so (a different structure is used to store this “big data”). As of this writing (in the summer of 2013), this issue does not appear to have been corrected; however, other tools, including RegRipper and the underlying Parse::Win32Registry Perl module have long since been updated to be able to correctly access these “big data” entries. This should clearly illustrate the fact that there’s a big difference between stating in your case notes that you “parsed/analyzed the Registry,” and specifically stating what tools you used to do so, and why doing so is important.
Over the past year or so, one data structure in particular has garnered a great deal of attention from a very small segment of the forensic analysis community; shell items. While Microsoft provides a good deal of documentation regarding structures that make use of the shell items, these structures are not themselves clearly defined, and as such, have only been identified by the tireless efforts of a few researchers within the community. These data structures are very important because they have actually existed on Windows systems for quite some time. For example, Microsoft’s shell link binary file format specification (found online at http://msdn.microsoft.com/en-us/library/dd871305.aspx) identifies the section of the Windows shortcut files that is comprised of shell items (more specifically, of a shell item ID list). Up until about two years ago, very few, if any, of the commonly used tools for parsing Windows shortcut/shell link (LNK) files parsed this shell item ID list, and instead went straight to the LinkInfo block. I have seen examples of legitimate Windows shortcut files during analysis that were comprised of just the shell item ID list and did not have a LinkInfo block, meaning that the commonly used tools would simply not show any output. This could be a critical piece of evidence—the shortcut was for a digital camera that had been connected to the system, and could lead to identifying a source of images. There have been examples of malware that have created Windows shortcut files consisting solely of shell item ID lists (no LinkInfo block), and there have been examples of how the shell item ID list can be manipulated (without modifying the LinkInfo block) to force the shortcut to execute something other than the intended target file. More recently, tools have been created to parse the shell item ID lists found in LNK files, as well as to parse other artifacts that include shell items. The point is that tools develop and evolve over time as new information becomes available, so identifying the name and version of the tool, as well as from where the tool was obtained, can be a very critical part of your case notes.
When I start my case notes, the first item I include, at the top of the page, are the goals of the analysis. This way, I have them right in front of me so that I can continually review them and keep focused. Too many times when conducting analysis, I’ve found interesting artifacts—Registry keys, log files, or entries in log files—that, while interesting, have not been pertinent to the goals of the examination. Spending too much time conducting research into these artifacts can significantly hamper my ability to complete the analysis in a thorough and timely manner. Putting the goals at the beginning of my case notes also allows me to review those goals each time that I open the case notes, such as beginning work again on subsequent days.
The next item I tend to incorporate into my case notes is a listing of the exhibits or items that I am analyzing. I identify from where I received the items, in what condition I received them, and any identifying marks or tags. For example, if I receive an image or log files on an external drive, I include the serial number of the drive. If I receive a laptop system and have to remove the hard drive in order to acquire an image, I will include identifying information about the laptop (serial number, asset tag number, etc.), a link to instructions that I use to remove the hard drive (many manufacturers make this information available online), and identifying information about the hard drive itself. I then include specific information regarding how the image was acquired, stating such things as whether I used a Tableau bridge and AccessData’s Forensic Toolkit (FTK) Imager, or if I employed a hardware write-blocker.
I generally keep my case notes on a daily basis, detailing what actions I took in my analysis, as well as the results. In some cases, I will even include specific excerpts from the output of tools that may direct or affect future analysis. While the goals of the exam will direct my analysis, a lot of the analysis work that I actually do is predicated in large part based on my experience, as is knowing what data may impact future analysis. As a consultant, I will also keep track of the number of hours that I work right there in my case notes. This can be very important if I’m working on several cases at once, as it is easier to keep track of this information for billing purposes.
Other information that may be critical to your case notes, particularly if it’s not maintained somewhere else, is point of contact information for your customer. Many times, it is important to maintain their full name, mailing and email address, phone number, etc.
Reporting
Reporting is often the most difficult aspect of digital analysis work. This is largely due to the fact that for many of us, communicating our findings in written form is not something that is necessarily included in our experiences, nor in our training. Personally, I do not have a great deal of formal training in digital analysis; with the exception of one vendor course in 1999, I have not attended any formal training, nor completed any coursework that is specific to this field. However, I have spoken to and worked with a number of analysts, and two of the common factors across all experience that they have shared with me has been that there are few courses that teach reporting, and that how reports are written and what they contain will vary not only between positions but also between managers. All of these factors combine to make report writing one of the most arduous and least “sexy” aspects of digital analysis.
So far in this chapter, we’ve addressed the role of goals in analysis, as well as how to document our analysis work in the form of case notes. By this point, the report should almost write itself. I mean, after all, we’ve got all of the components necessary for the actual report, and all we need to do is assemble them.
Format
The format you use for your reports can vary depending upon your audience. You may be in a portion of the industry with a law enforcement focus, and as such, the format you use for reporting may be different from what is used by a consultant communicating findings to a customer, or an analyst working for a company reporting their findings to the company’s HR director. My hope is that there are elements of and information shared in this chapter that are useful to you, regardless of the portion of the industry in which you operate.
When I was part of an emergency incident response team, we used one report format for our consulting work, and another for our Payment Card Industry (PCI) forensic assessment reports. While these formats were vastly different, there were both good and bad aspects of each. In particular, there were some aspects of the PCI report format that were very useful, and those are aspects that I’ll mention. These aspects of the format were useful enough that I’ve used them again and again in the various positions I’ve held in my career.
Executive summary
The executive summary of the report should be just that—a summary, for executives. Executives are generally not interested in the minutiae and details of an examination; more often than not, their concerns and interest are more closely aligned with the business itself. The executive summary should be no more than two pages, max, and be able to stand completely on its own. It’s useful, when writing the executive summary, to imagine that someone is going to tear it off of the report, and hand it to someone else. This is what I mean when I say that the executive summary should be able to stand on its own—it should be able to provide enough information to senior management to be able to address their concerns when separated from the body of the report.
I generally begin the executive summary with a brief description of the background of the incident, providing enough detail to make it clear as to why I or my employer was contacted by the customer in the first place. An example of a background statement might appear as follows:
On 21 July 2011, ACME Corporation (ACME) contacted the Example Consulting Team (ECT) regarding a potential computer security incident, thought to include compromise of and intrusion into the ACME computing infrastructure. ACME had reportedly been contacted by a federal law enforcement agency regarding information specifically associated with ACME being found on a web site external to ACME.
Following the background statement, I then clearly list the goals of the examination. If the goals are concise and limited, this section might simply be a sentence. For example, consider a case involving a corporate HR complaint, the goal statement in the executive summary may appear as follows:
The goal of this exam was to determine if a specific user had violated corporate acceptable use policies by accessing unauthorized web sites.
For multiple goals, this statement may appear as “The goals of the examination were as follows:”, followed by a numbered bullet list of the goals. As described previously in this chapter, it is important that the goals be clear and concise, and broken down into achievable tasks. This way, each individual goal can be addressed as a specific task to be accomplished through analysis of the available data.
The conclusions listed in the executive summary should directly follow the goals; that is to say that for each goal, you should have an associated conclusion. Generally speaking, your conclusions will likely be one of three different responses; yes, no, or there wasn’t enough information available to determine an accurate response. For example, your conclusion for the HR goal listed previously might be (analysis performed by Acme Consulting):
Acme Consulting was able to determine that the user did, in fact, violate corporate acceptable use policies by visiting unauthorized web sites.
What web sites are determined to be “unauthorized” may be detailed in the HR policy handbook, the acceptable use policies signed by the user, or they may have been provided by the HR director during the course of the analysis. A similar approach would apply for other analysis goals, such as whether a user accessed resources (file shares) or documents for which they were not authorized. The conclusions statement above can further include the dates of the activity, or those dates can be included in an attachment or appendix to the report.
Next, let’s consider a malware incident, which may be a bit more expansive and include additional goals. The goals of such an exam may be listed in the executive summary as follows:
The goals of this examination were as follows:
1. Determine if the system in question was infected with malware.
2. If the system was found to have been infected with malware, determine if that malware was capable of collecting and exfiltrating information from the system.
3. Determine what data, if any, was exfiltrated from the system by or through the use of the malware.
Following the goals, the conclusions (again, the analysis was performed by the ECT) may appear as follows:
Through detailed and thorough analysis, Acme Consulting was able to determine the following:
1. The system was in fact infected with malware; the malware was identified as<insert name>.
2. The identified malware was found to be written specifically to collect authentication information from the infected system, and then exfiltrate that information to a compromised system.
3. The ECT was unable to definitely determine data that may have been exfiltrated from the system by or through the use of the malware, as neither full network traffic captures nor detailed network session logs were available for analysis.
Again, these goals and conclusions are notional, and the point has been to illustrate how the conclusions should be as concise and clear as the goals, and directly associated with each of the goals. For each analysis goal, there is an associated conclusion, so that anyone reading the report, even weeks or months after the fact, can clearly see what was asked, and then see what was found or determined. This means of reporting is clear, concise, and simple.
Body
The body of the report goes well beyond the executive summary, and contains the details of the analysis conducted and the associated findings. Some aspects of the report may require, due to size, that they be incorporated into an appendix or attachment to the report.
Background
At the beginning of the body of the report, I have found it valuable to also include a description of the background of the incident, or the reason for the analysis, similar to what was included in the executive summary. What I tend to do is copy the background from the executive summary, paste it into the body of the report, and then add additional detail for clarity, as necessary. Again, I tend to assume that the executive summary will be removed from the actual report, and this methodology ensures consistency in the overall report.
Throughout the entire report, and particularly in the background information found in the body of the report, I tend to not refer to specific individuals by name, unless specifically asked to do so. I refer to customers by their organizational name, and depending upon the circumstances, I may refer to individuals by title. I have found that this tends to avoid confusion, particularly when engaging with a large organization. If every name of every individual with whom I had communicated and received information or exhibits were listed, the point of the report would be lost. Further, confusion can result from simple disparities in communications, such as when stating that “…the laptop was shipped by John Doe, the ACME Corporation Director of IT…,” and Mr. Doe had, in fact, tasked someone from the shipping department ensure that the item in question was safely packed and shipped. It is much simpler to state, “ACME Corporation provided the identified laptop computer system for analysis….”
In addition to the background information, I will also include a clear description of the provided items or exhibits, taken directly from my case notes. For a relatively small number of exhibits, a short paragraph or bullet list will suffice; for a larger number of exhibits, a table or even an appendix to the report might be more appropriate. Regardless, keeping a detailed list of exhibits or items provided for analysis lends a modicum of context to the report, by clearly identifying what data sources were available. This way, there’s little confusion as to what was available for analysis, and ultimately from where the conclusions originated.
Analysis
The analysis section of the report easily contains the most technical detail in the report; after all, this section of the report essentially embodies the customer contacted and engaged you in the first place.
I find it beneficial to start the analysis findings section of the report by clearly stating the goals of the analysis. This information is duplicated directly from (or, depending upon the order in which you write the report, to…) the executive summary, in order to maintain consistency in the report. This also makes it much easier to write the report overall, as copy and paste is far simpler than having to think up another way of saying the same thing. Further, this helps keep me focused when it comes to writing the report, in the same manner as including the analysis goals at the beginning of my case notes keeps my analysis focused. By including the goals at the beginning of the analysis section, I not only keep my own analysis focused as I write this section of the report, but it also serves to help focus the attention of the reader, as well.
After adding the goals to the report, I then detail my analysis and findings. Most of this is taken directly from my case notes, albeit in perhaps a more organized and consistent manner. What I mean by this is that regardless of the sequence in which I conducted my analysis process (as detailed in my case notes), I want to ensure that this section flows smoothly to my conclusions. As such, I may need to present some of my findings out of order, with respect to when I actually conducted the analysis. For example, if, as detailed in my cases notes, I began my analysis on day 1 and then opted (or was required) to perform a scan or keyword search, and on the second day of analysis, made another finding, in my report I would not necessarily make the distinction as to when the actual analysis had been conducted. It may be far more beneficial to list the findings in succession, so that they crafted a better understanding and flowed into the conclusion in a manner that makes better sense. The order of how you arrived at your findings is not as important as communicating those findings in a manner that leads logically to your conclusions.
An example of an analysis findings statement from a malware examination might appear as follows:
The Example Consulting Team (ECT) mounted the acquired image as a read-only volume (via AccessData’s FTK Imager version 3.1.3), and scanned the volume using<name and version of AV product>, which had been updated immediately prior to the conducting the scan.<AV product name>identified several cookies associated with a previously-used user profile on the system, but no malware was identified.
Had the antivirus product used to scan the mounted volume identified malware within the volume, this fact would be reflected in the statement along with the name of the malware provided by the product. This can be very important, as various vendors often refer to the same malware with different names.
Another example of an analysis statement included in this section of the report might appear as follows:
The Security hive file found in the Windows\system32\config folder within the acquired image was parsed using the auditpol.pl plugin (plugin version 20121128) included in the 18 April 2013 plugin archive, for RegRipper version 2.8. The output of the plugin indicated that auditing was not enabled, and the LastWrite time for the PolAdtEv Registry key indicated that it had last been modified more than three years before the date of the incident. This finding indicates that the audit configuration on the date of the incident was not the result of actions taken by the intruder, but instead part of configuration policies associated with provisioning the system for use by the employee. Parsing the Security hive files located in the available System Restore Points via the use of ripxp.exe (uploaded to the RegRipper Google Code site on 29 June 2013) produced consistent findings of the same PolAdtEv key LastWrite time.
Notice that the elements of this analysis statement is consistent with the one listed previously in this chapter; the data being analyzed and the tool used to access or parse the data, as well as the version of the tool, is clearly identified, as are the results, and the analyst’s findings based on those results. This information can be taken directly from the analyst’s case notes. The analyst might follow the above statement with one regarding analysis of the Event Logs available on the system, which might appear as follows:
The Security, System, and Application Event Logs (secevent.evt, sysevent.evt, and appevent.evt, respectively) were exported from the Windows\system32\config folder within the acquired image, and each file was parsed using evtparse.exe (no version information was provided by the tool). The Security Event Log was found to not include any event records, which is consistent with the previous finding regarding the audit configuration of the system. This was verified manually by opening the Event Log file in a hex editor and after skipping over the 48-byte header of the file, observing that the remainder of the 512 kilobyte file did not contain any valid event records.
Notice the flow of how the analysis is presented in these two statements. It demonstrates a logical flow of analysis from one artifact to another, and assists in building the foundation for your conclusions. Further, these statements provide enough detail that another analyst, perhaps one of your customer’s employees, can validate your findings should any questions arise.
I firmly believe that is important to provide not just the portions of your analysis where you actually find something, but you should also include those analysis actions that you take that do not produce any (or, the expected) results. The reason is that this illustrates a thorough approach to your analysis, and tells the reader that, yes, you not only considered a particular avenue of analysis, but you pursued it and the result was that nothing was found. Including this information in your report helps you to answer the overall question, “how did you arrive at your conclusions?”
Conclusions
You should complete the body of your report with a reiteration of the conclusions listed in the executive summary; I tend to do this by duplicating these directly from the executive summary, or vice versa (you may opt to write the body of the report first, and then duplicate the conclusions in the executive summary). Again, each of the listed goals of the analysis should have a corresponding conclusion; if, as one of the goals of your analysis, you were asked to determine if the system had been infected with malware, you should have an associated conclusion that flows directly from your analysis findings that addresses this goal directly.
Writing tips
I have found over time that it can be very useful to keep a few simple tips in mind when writing a report. One of perhaps the most useful aspects of the PCI report format specified by Visa (during the time that I was conducting this type of examination) was consistency. Information and analysis regarding how someone went about stealing credit card numbers can be extremely technical, but the audience to whom the report is intended—the affected merchant, the bank, and the staff at Visa—are more adept in business areas than the world of bits and bytes. As such, maintain a consistent writing style in the report was invaluable in communicating technical finding and conclusions to those readers.
Throughout my (at this writing) over fourteen years working in incident response and digital analysis, one thing that I see again and again is that no matter how efficient and thorough an analyst may be, they are most often hung up on the report writing aspect of the job. This is very often true even when there is a very clear format, or even a detailed template, available. This can be due to the fact that as the analyst is writing the findings section of their report, they are looking for new and different ways to describe or explain their analysis steps, and what they did. Noodling over the prose of a report is not productive. For one, it wastes time. How many different ways are there to say, “…I took this data, ran this tool to parse and display information derived from that data, and based on the output displayed by the tool, here are my findings”? Remaining consistent in writing your findings, as well as how the goals and conclusions are presented in different part of the report, help to clearly communicate your findings, as well as minimize confusion on the part of the reader.
Not long after I got out of the military, I was working for a company where the predominance of my job was to conduct vulnerability assessments. We did a lot of vulnerability assessments, in part, because we were good at them, but our volume of work had in large part to do with the fact that our team salesman could sell ice cubes to someone living in northern Alaska, and leave them feeling like they’d gotten a deal. As we did more and more work, we started to see that we were seeing a lot of the same sorts of vulnerabilities and issues within the various organizations that we visited, and it often turned out that we’d reuse paragraphs from previous reports, often tweaking them slightly to meet the needs of the particular customer or assessment. This was especially true if someone on our team had come up with a truly magical combination of words that ideally suited that particular finding or aspect of the report. Honestly, if the findings were the same or similar enough to require minimal changes, then why rewrite the entire section of the report when copy and paste, perhaps with a little bit of tweaking, will do just fine? We instead opted to save time by not trying to come up with new ways to say the same things.
Okay, so what does this have to do with the PCI reports? The format was specified and provided to us, and applied to all of the forensic assessments that we did; the format was to be used in each and every engagement, which meant that reports between customers would be pretty consistent. That made it easy, as there was little guess work; we knew exactly which tool to use for various sections of the report, the reporting format provided to us specified that certain actions (searches for magnetic stripe or “track” data, keyword, as well as file name and file hash searches, etc.) were to be performed during every engagement. In the section of the report that addressed the analysis work that was actually performed, each subsection was similarly consistent. We would describe the tool and version that we used, what we did, and provide a description of our findings. Over and over. This may sound monotonous, but rather than being boring, it did two things for us. First, it made the report easier and quicker to write, because all you had to do was go to your case notes, and transpose the information over to the report. Include the tool name, version, what was done, what you found—all of this was taken directly from the analyst’s case notes, and the Findings section of the report pretty much wrote itself.
Second, being consistent simply made it easier to communicate technical information to less- or nontechnical people—such as our customers. A reader from say, the merchant organization or the bank, is adept at what they do, and we were hired for our expertise at performing these examinations. That isn’t to say that our customers weren’t smart, but we were dealing with extremely technical information which our customers needed to understand in order to make critical business decisions. What I saw was that if we weren’t consistent, then our customers didn’t so much have questions regarding the technical information that we included in the report; rather, the questions most often pertained to why things were different. What I saw over time was that when trying to communicate technical and potentially unfamiliar information, it’s best to be consistent in your formatting and phrasing; otherwise, the reader picks up on the differences, rather than the technical content, or what you’re really trying to communicate to them.
I also found it useful, even valuable, to stick with plain, simple language. Don’t try to embellish things, as this can lead to confusion. I’m as guilty as anyone else (perhaps even more so) of at times trying to use flowery language to express myself, in the hopes that it conveys professionalism and credibility. I’ve found more often than not that this can also lead to confusion, particularly if the person reading the report (in hopes of gleaning the necessary understanding to make a business or legal decision) either doesn’t understand the terminology being used, or has a different understanding of it all together. If you parsed data from a Registry hive file using a particular RegRipper plugin, or by opening the hive file in a viewer and simply navigating to the key or value in question, just say that. If you did the same thing for other Registry keys or values, simply say so. This approach to writing your report reduces ambiguity and confusion.
Peer review
One aspect of reporting of which I have seen very little in the 25 or so years that I have been involved in information security, and the 14+years that I have been specifically involved in digital forensics and incident response, is peer review of what’s being written, in particular case notes and reports. Now, I know that not everyone has the luxury of time for conducting peer review, as time is often not on our side. As a consultant, it’s always important to deliver your findings to your customer in a timely manner; however, this does obviate the opportunity for and value of peer review at a later date. Most often, the review and scrutiny to which my case notes and reports have been subject has been conducted by my manager, before the final report was sent to the customer. Having those same items reviewed by my peers, who are doing the same sort of work that I’m doing every day, can lead to all of us learning new things and improving our analysis processes.
Peer review, while a critical component to establish the credibility of an author in other fields (particularly academia), seems to only be conducted within the digital analysis field by those analysts who actively seek self-improvement. Don’t get me wrong—I know that reports cannot be shared outside of organizations, and that this is especially true for law enforcement. But what about within an organization? If you’re a member of team of analysts, have you asked another analyst to review your case notes, to see if you’ve missed anything? Have you asked another team member to review a report before you sent it to your manager?
I’ve been told that within law enforcement, at all levels, reports are reviewed by a supervisor before being accepted as a final report. Often, this review will reveal that additional information or analysis is required, and the report will be updated. At a minimum, the supervisor ensures that that elements of the crime are articulated in the report.
Summary
Analysts will very often tell you what that what they enjoy most about their chosen profession is the analysis, and that what they enjoy the least is report writing. I can honestly say that until I “cracked the code” and understood what this one particular manager expected to see in reports that he approved for final delivery to customers, I felt the same way. Like many analysts, at first, I struggled with report writing; however, once I began to see patterns in what this manager was looking for, it got easier. Eventually, I got to the point where I would send a report draft to this manager, and within a couple of hours (or by the following morning) receive approval to “go final” and follow our process for final report delivery. I hope that in this chapter, I’ve been able to convey the accumulated knowledge of what I learned, not only from working with that manager but also as a result of the breadth of various writing assignments I’ve had throughout my career, and that this information is helpful to you in some way. After all, the smartest, most capable analyst is ineffective if they are unable to communicate their findings.