Chapter 13

Reporting

Abstract

The various types of assessment reports are defined and reviewed. The Security Assessment Report and the Risk Assessment Report are the primary outputs from any assessment in the RMF process and each is defined and discussed, and sample templates are provided.

Keywords

reports
SAR
POAM
I often explain to interested people and my students that the number 1 job of any security professional is the secure the data and the number 2 job is to “report, report, and report again.” We often have to spend a great deal of time and effort in gathering the data for and producing various different kinds of reports and documents to support our security efforts. The various reporting requirements often are externally provided to the assessor via compliance needs, external agency needs, or internal department needs. Each component needing the assessor report has a distinct and, I hope, clearly defined method for reporting along with the data and results necessary for the report to contain:
1. SAR
2. Rescue and Recovery (RAR)
3. Artifacts as reports
4. Public Inquiry and Response (PIAR)
5. Remediation actions
6. Plant Operation and Maintenance (PO&Ms)

The primary purpose of the security and privacy assessment reports is to convey the results of the security and privacy control assessments to appropriate organizational officials. The security assessment report is included in the security authorization package along with the security plan (including an updated risk assessment), and the plan of action and milestones to provide authorizing officials with the information necessary to make risk-based decisions on whether to place an information system into operation or continue its operation. Organizations may choose to include similar privacy-related artifacts in the authorization package to convey essential information to authorizing officials. All issues associated with compliance to privacy-related legislation, directives, regulations, or policies are coordinated with the Senior Agency Official for Privacy (SAOP)/Chief Privacy Officer. As the assessment and authorization process becomes more dynamic in nature, relying to a greater degree on the continuous monitoring aspects of the process as an integrated and tightly coupled part of the system development life cycle, the ability to update the security and privacy assessment reports frequently becomes a critical aspect of information security and privacy programs.

It is important to emphasize the relationship, described in Special Publication 800-37, among the three key documents in the authorization package (i.e., the security plan, the security assessment report, and the plan of action and milestones). It is these documents that provide the most reliable indication of the overall security state of the information system and the ability of the system to protect to the degree necessary, the organization’s operations and assets, individuals, other organizations, and the Nation. Updates to these key documents are provided on an ongoing basis in accordance with the continuous monitoring program established by the organization. Updates to similar privacy-related documents occur at a frequency and format determined by the SAOP in coordination with authorizing officials.

The security and privacy assessment reports provide a disciplined and structured approach for documenting the findings of the assessor and the recommendations for correcting any weaknesses or deficiencies in the security and privacy controls. This appendix provides a template for reporting the results from security and privacy control assessments. Organizations are not restricted to the specific template format; however, it is anticipated that the overall report of an assessment will include similar information to that detailed in the template for each security and privacy control assessed, preceded by a summary providing the list of all security and privacy controls assessed and the overall status of each control.1

Key elements for assessment reporting

The following elements are included in security and privacy assessment reports:
Information system name
Security categorization
Site(s) assessed and assessment date(s)
Assessor’s name/identification
Previous assessment results (if reused)
Security/privacy control or control enhancement designator
Selected assessment methods and objects
Depth and coverage attributes values
Assessment finding summary (indicating satisfied or other than satisfied)
Assessor comments (weaknesses or deficiencies noted)
Assessor recommendations (priorities, remediation, corrective actions, or improvements)

The assessment findings

Each determination statement executed by an assessor results in one of the following findings:
1. Satisfied (S)
2. Other than satisfied (O)
During an actual security and privacy control assessment, the assessment findings, comments, and recommendations are documented on appropriate organization-defined reporting forms. Organizations are encouraged to develop standard templates for reporting that contain the key elements for assessment reporting described above. Whenever possible, automation is used to make assessment data collection and reporting cost-effective, timely, and efficient.

Security Assessment Report

The results of the security control assessment, including recommendations for correcting any weaknesses or deficiencies in the controls, are documented in the security assessment report. The security assessment report is one of three key documents in the security authorization package developed for authorizing officials. The assessment report includes information from the assessor necessary to determine the effectiveness of the security controls employed within or inherited by the information system based upon the assessor’s findings. The security assessment report is an important factor in an authorizing official’s determination of risk to organizational operations and assets, individuals, other organizations, and the Nation. Security control assessment results are documented at a level of detail appropriate for the assessment in accordance with the reporting format prescribed by organizational and/or federal policies. The reporting format is also appropriate for the type of security control assessment conducted (e.g., developmental testing and evaluation, self-assessments, independent verification and validation, independent assessments supporting the security authorization process or subsequent reauthorizations, assessments during continuous monitoring, assessments subsequent to remediation actions, independent audits/evaluations).

Security control assessment results obtained during system development are brought forward in an interim report and included in the final security assessment report. This supports the concept that the security assessment report is an evolving document that includes assessment results from all relevant phases of the system development life cycle including the results generated during continuous monitoring. Organizations may choose to develop an executive summary from the detailed findings that are generated during a security control assessment. An executive summary provides an authorizing official with an abbreviated version of the assessment report focusing on the highlights of the assessment, synopsis of key findings, and/or recommendations for addressing weaknesses and deficiencies in the security controls.2

Here is a governmental-based SAR-formatted Table of Contents for use and suggestions:

Executive summary

Introduction
Risk Summary
Conclusion
1 INTRODUCTION
1.1 Scope
1.2 Background
1.3 Assessment Methodology
1.3.1 Likelihood Determination
1.3.2 Impact Determination
1.3.3 Risk Scale
1.3.4 Mitigation Actions
1.4 Assumptions and Constraints
2 SYSTEM DESCRIPTION
3 THREAT STATEMENT
4 SECURITY ASSESSMENT RESULTS
4.1 Automated Scan Results
5 STATEMENT OF WEAKNESSES
5.1 Management Controls
5.2 Operational Controls
5.3 Technical Controls
5.4 Risk Based Decision Recommendation
6 SUMMARY AND SIGNATURES
Appendix A: Detailed Security Assessment Results
This SAR covers the full scope of the SCA activities defined in SP 800-37 and in this book, so the report should reflect all of the activities the SCA took during the assessment.

Risk Assessment Report

The essential elements of information in a risk assessment can be described in three sections of the risk assessment report (or whatever vehicle is chosen by organizations to convey the results of the assessment):

(i) An executive summary;
(ii) The main body containing detailed risk assessment results; and
(iii) Supporting appendices.
A. Executive Summary
List the date of the risk assessment.
Summarize the purpose of the risk assessment.
Describe the scope of the risk assessment.
- For Tier 1 and Tier 2 risk assessments, identify: organizational governance structures or processes associated with the assessment (e.g., risk executive [function], budget process, acquisition process, systems engineering process, enterprise architecture, information security architecture, organizational missions/business functions, mission/business processes, information systems supporting the mission/business processes).
- For Tier 3 risk assessments, identify: the information system name and location(s), security categorization, and information system (i.e., authorization) boundary.
State whether this is an initial or subsequent risk assessment. If a subsequent risk assessment, state the circumstances that prompted the update and include a reference to the previous Risk Assessment Report.
Describe the overall level of risk (e.g., Very Low, Low, Moderate, High, or Very High).
List the number of risks identified for each level of risk (e.g., Very Low, Low, Moderate, High, or Very High).
B. Body of the Report
Describe the purpose of the risk assessment, including questions to be answered by the assessment. For example:
- How the use of a specific information technology would potentially change the risk to organizational missions/business functions if employed in information systems supporting those missions/business functions; or
- How the risk assessment results are to be used in the context of the RMF (e.g., an initial risk assessment to be used in tailoring security control baselines and/or to guide and inform other decisions and serve as a starting point for subsequent risk assessments; subsequent risk assessment to incorporate results of security control assessments and inform authorization decisions; subsequent risk assessment to support the analysis of alternative courses of action for risk responses; subsequent risk assessment based on risk monitoring to identify new threats or vulnerabilities; subsequent risk assessments to incorporate knowledge gained from incidents or attacks).
Identify assumptions and constraints.
Describe risk tolerance inputs to the risk assessment (including the range of consequences to be considered).
Identify and describe the risk model and analytic approach; provide a reference or include as an appendix, identifying risk factors, value scales, and algorithms for combining values.
Provide a rationale for any risk-related decisions during the risk assessment process.
Describe the uncertainties within the risk assessment process and how those uncertainties influence decisions.
If the risk assessment includes organizational missions/business functions, describe the missions/functions (e.g., mission/business processes supporting the missions/functions, interconnections and dependencies among related missions/business functions, and information technology that supports the missions/business functions).
If the risk assessment includes organizational information systems, describe the systems (e.g., missions/business functions the system is supporting, information flows to/from the systems, and dependencies on other systems, shared services, or common infrastructures).
Summarize risk assessment results (e.g., using tables or graphs), in a form that enables decision makers to quickly understand the risk (e.g., number of threat events for different combinations of likelihood and impact, the relative proportion of threat events at different risk levels).
Identify the time frame for which the risk assessment is valid (i.e., time frame for which the assessment is intended to support decisions).
List the risks due to adversarial threats (see Table F-1).
List the risks due to non-adversarial threats (see Table F-2).
C. Appendices
List references and sources of information.
List the team or individuals conducting the risk assessment including contact information.
List risk assessment details and any supporting evidence (e.g., Tables D-7, D-8, E-5, F-3, F-6, H-4), as needed to understand and enable reuse of results (e.g., for reciprocity, for subsequent risk assessments, to serve as input to Tier 1 and Tier 2 risk assessments).3
With the redesign of SP 800-30, revision 1, the RAR is not always a deliverable item for RMF activities, but often is used to supplement ongoing remediation efforts and major change analysis efforts for system. Additionally, RAR process reports are used when threat environments or operational environments are changed with respect to the system under review.

Artifacts as reports

Often the development and production of assessment artifacts creates reports and deliverables of their own, and these documents become part of the supplemental and supporting information that goes with the authorization package to the Authorizing Official (AO) and Authorizing Official Designated Representative (AODR) for review and approval. Items such as Vulnerability Scan Reports, Configuration Change Request Installation Reports, and Contingency Plan Test Results all are used in both cases: as support for authorization package and as security report deliverables.

Privacy impact assessment report

Privacy reporting usually entails two different kinds of reporting criteria. First type involves the actual reporting of the Personally Identifiable Information (PII) on the system under review. Types of data collected and retained and their security and privacy requirements are defined in these types of reports. The second type of report is the status reporting needed for governmental agencies to give external organizations, such as Congress, a view of PII and privacy in each agency.
The format and data requirements for Privacy Assessment Reports are agency and governance based and vary for each agency. If you have any questions, typically you should contact your agency Privacy Office, since federal law requires each agency to have such an office.
Remember, Appendix J of SP 800-53, rev. 4, has a listing of all the privacy controls and their requirements now mandated for all agencies by OMB.

Remediation efforts during and subsequent to assessment

The security assessment report provides visibility into specific weaknesses and deficiencies in the security controls employed within or inherited by the information system that could not reasonably be resolved during system development or that are discovered post-development. Such weaknesses and deficiencies are potential vulnerabilities if exploitable by a threat source. The findings generated during the security control assessment provide important information that facilitates a disciplined and structured approach to mitigating risks in accordance with organizational priorities. An updated assessment of risk (either formal or informal) based on the results of the findings produced during the security control assessment and any inputs from the risk executive (function), helps to determine the initial remediation actions and the prioritization of such actions. Information system owners and common control providers, in collaboration with selected organizational officials (e.g., information system security engineer, authorizing official designated representative, chief information officer, senior information security officer, information owner/steward), may decide, based on an initial or updated assessment of risk, that certain findings are inconsequential and present no significant risk to the organization. Alternatively, the organizational officials may decide that certain findings are in fact, significant, requiring immediate remediation actions. In all cases, organizations review assessor findings and determine the severity or seriousness of the findings (i.e., the potential adverse impact on organizational operations and assets, individuals, other organizations, or the Nation) and whether the findings are sufficiently significant to be worthy of further investigation or remediation. Senior leadership involvement in the mitigation process may be necessary in order to ensure that the organization’s resources are effectively allocated in accordance with organizational priorities, providing resources first to the information systems that are supporting the most critical and sensitive missions and business functions for the organization or correcting the deficiencies that pose the greatest degree of risk. If weaknesses or deficiencies in security controls are corrected, the security control assessor reassesses the remediated controls for effectiveness. Security control reassessments determine the extent to which the remediated controls are implemented correctly, operating as intended, and producing the desired outcome with respect to meeting the security requirements for the information system. Exercising caution not to change the original assessment results, assessors update the security assessment report with the findings from the reassessment. The security plan is updated based on the findings of the security control assessment and any remediation actions taken. The updated security plan reflects the actual state of the security controls after the initial assessment and any modifications by the information system owner or common control provider in addressing recommendations for corrective actions. At the completion of the assessment, the security plan contains an accurate list and description of the security controls implemented (including compensating controls) and a list of residual vulnerabilities.4

POAMs

Once the Security Assessment Report is provided to the system owner and the AODR, the system owner develops the plans for remediation of the residual risk items identified by the SAR. These items are placed onto the Risk Register for the agency, known as the Plan of Action and Milestones (POAMs).

The plan of action and milestones, prepared for the authorizing official by the information system owner or the common control provider, is one of three key documents in the security authorization package and describes the specific tasks that are planned:

(i) To correct any weaknesses or deficiencies in the security controls noted during the assessment; and
(ii) To address the residual vulnerabilities in the information system.

The plan of action and milestones identifies:

(i) The tasks to be accomplished with a recommendation for completion either before or after information system implementation;
(ii) The resources required to accomplish the tasks;
(iii) Any milestones in meeting the tasks; and
(iv) The scheduled completion dates for the milestones.

The plan of action and milestones is used by the authorizing official to monitor progress in correcting weaknesses or deficiencies noted during the security control assessment. All security weaknesses and deficiencies identified during the security control assessment are documented in the security assessment report to maintain an effective audit trail. Organizations develop specific plans of action and milestones based on the results of the security control assessment and in accordance with applicable laws, Executive Orders, directives, policies, standards, guidance, or regulations. Plan of action and milestones entries are not required when weaknesses or deficiencies are remediated during the assessment or prior to the submission of the authorization package to the authorizing official.

Organizations define a strategy for developing plans of action and milestones that facilitates a prioritized approach to risk mitigation that is consistent across the organization. The strategy helps to ensure that organizational plans of action and milestones are based on:

(i) The security categorization of the information system;
(ii) The specific weaknesses or deficiencies in the security controls;
(iii) The importance of the identified security control weaknesses or deficiencies (i.e., the direct or indirect effect the weaknesses or deficiencies may have on the overall security state of the information system, and hence on the risk exposure of the organization, or ability of the organization to perform its mission or business functions); and
(iv) The organization’s proposed risk mitigation approach to address the identified weaknesses or deficiencies in the security controls (e.g., prioritization of risk mitigation actions, allocation of risk mitigation resources). A risk assessment guides the prioritization process for items included in the plan of action and milestones.5
POAM reporting is defined by OMB through its memoranda and currently is required for agency on a monthly basis. It had run on a quarterly basis for many years, but recent events have moved OMB to request these reports through CyberScope to be submitted on a monthly basis for budgetary and management purposes.