CHAPTER 11

TEST OF CONTROLS, TRANSACTIONS, AND ACCOUNTS

Audit procedures are designed to gather evidence and should be viewed as processes that continuously seek to obtain and evaluate sufficiently appropriate audit evidence. However, in practical terms, the bulk of appropriate audit evidence supporting the auditor’s financial statement assertions is the result of testing conducted during Phase III of the Federal Government’s audit model. The testing phase encompasses both internal control and substantive testing.

Throughout the audit process, the objective is the continual assessment and validation of the nature of audit risk, which, in its simplest form, is the risk of the auditor unknowingly issuing a clean audit opinion when the financial statements are materially misstated. AU-C 200 states that audit risk encompasses:

• Inherent risks relate to the susceptibility of a process or account to be misstated. For example, estimates such as those required by the Federal Credit Reform Act require complex calculations that are more likely to include errors, than, for example, simple payroll accrual estimates at the end of a reporting period. Similarly, certain assets, such as cash, are more susceptible to theft than others. Finally, certain assets are subject to spoilage unless consumed/disposed of on a timely basis. Inherent risks also arise from factors external to the agency such as changes in congressional funding levels and/or congressionally mandated change in the program’s mission. Changes such as these could affect the value of, or render obsolete, assets such as inventories and other property that the agency had created for future use, but which was rendered valueless or nearly so as a result of the change in funding or program.
• Control risks relate to the possible occurrence of material misstatements that are not prevented or detected on a timely basis by the auditee’s internal control processes. In other words, for some reason(s), the “prescribed” internal controls failed to work. Control risk also includes accounts, processes, cycles, or systems that are susceptible to certain threats that are not mitigated by the presence of effective internal control practices and procedures that reduce the likelihood of the threat to a reasonably low level. (Control risks cannot be reduced to zero, given the inherent limitations of any system of internal controls.)
• Detection risks relate to the auditor’s failure to detect a misstatement. A sound audit approach executed by competent personnel with the requisite skills and experience necessary for the tests included in the approach should reduce this risk to a reasonable level. Since the auditor does not examine 100% of all transactions, this risk is always present. However, it is the auditor’s responsibility to develop a sound audit approach to minimize detection risk.

Frequently, careful and informed interviews of agency executives, program managers, financial personnel, and data processing staffs provide early clues to client attitudes, pressures, problems, contingencies, and high-risk areas not readily apparent from examinations of general ledger (GL) accounts or not noted in tests of detailed transactions. Throughout the audit, but particularly in regard to the testing described in this chapter, the audit team must continually reflect on the impact of projected misstatements, not just discoveries and correction of specific errors.

Exhibit 11.1 highlights several aspects and issues concerning testing applications that should, and in some cases must, be used when conducting an audit pursuant to Government Auditing Standards (GAS; the Yellow Book).

EXHIBIT 11.1 Test of Controls, Transactions, and Accounts

TYPES OF TESTS

Audit testing traditionally takes one of two forms: internal control or substantive testing. In government auditing, Federal guidance provides for a third type of testing: testing of compliance with laws and regulations.

Tests of Controls

Tests of internal controls are designed to verify whether the internal controls identified during the planning phase of the audit, in the auditor’s evaluation of the system design, are, in fact, operating and/or being complied with in practice. As such, a test of internal controls is appropriate only (and can be effective only) if, in the auditor’s judgment, the control being tested can effectively detect or prevent an event or transaction that could negatively affect important assertions.

Substantive Tests

Substantive tests are designed to validate an account balance by direct validation of the account (e.g., physical observation of property, review of documentation such as invoices to verify assigned costs, direct confirmation of balances with third parties, etc.) or by developing an independent estimate of what the account balance should be (e.g., by applying statistical techniques or executing analytical procedures) and comparing with the data recorded in the auditee’s records. While the auditor may determine that a substantive test is a more practical means of obtaining appropriate audit evidence, he or she must still assess the impact of internal controls on assertions, transactions, or accounts that are subjected only to substantive audit procedures.

Tests of Compliance with Laws and Regulations

Noncompliance with Federal financial laws and regulations may have a direct impact on an entity’s financial statements. For example, noncompliance with an environmental law may result in a contingent or possibly actual liability that needs to be recorded or disclosed according to the specific circumstances. In this respect, governmental auditing is similar to commercial auditing, and this type of compliance can be considered in connection with specific internal controls and the internal control environment. However, the Office of Management and Budget (OMB) and the agency itself will also identify certain laws and regulations that may not have a material impact on the financial statements of the agency but that nevertheless should be considered by the auditor. It is important for the auditor to be aware of these regulations and to test compliance, as required by the guidance.

TESTING INTERNAL CONTROLS

Tests of controls require the application of audit procedures with the objective of validating (or disproving) the operational effectiveness of the internal controls. The test is designed to determine whether controls are, in fact, operating as designed. To the extent that testing confirms that controls are operating as described in the auditor’s documentation of the system design, the auditor obtains the necessary support to validate the initial control risk assertion (e.g., level of assurance) and is in a position to continue with the execution of the audit approach developed during the internal control phase.

No definition of operational effectiveness exists in law, OMB or Treasury Department regulations, the accounting and audit guidance of the Government Accountability Office (GAO), or the generally accepted accounting standards (GAAS) of the American Institute of Certified Public Accountants (AICPA). However, auditors must establish the operational effectiveness of controls if reliance on internal controls is to serve as a tool in support of the audit assertions. In order to conclude that controls are operationally effective, auditors must consider whether the internal controls are:

• An effective deterrent against theft, misuse, waste, or accidental destruction of physical assets (e.g., property, plant, and equipment) and nonphysical assets (e.g., financial and nonfinancial systems, software, other records supportive of financial and operational management and reporting).
• An effective safeguard over the collection, recording, processing, and ultimate reporting of financial and nonfinancial data (regardless of whether the systems are manual or electronic in nature, the information is recorded at remote sites or centrally, by professional or nonprofessional, financial or nonfinancial personnel, at the auditee’s sites or at outsourced locations).
• Effectively providing reasonable assurance of prevention or detection for instances of noncompliance with laws, regulations, contracts, and grants that have a direct and material effect in determining financial statement amounts or other data significant to the audit objectives.

Conversely, where tests disclose that controls are not effective or were not fully operational throughout the year, the auditor must revise the audit approach. In other words, negative or unsatisfactory test results oblige the auditor to reassess the evaluation of internal control risk. To the extent that the auditor reduces the reliance on internal controls, he or she must revise the audit approach accordingly.

Typically, when the initial assessment of control risk is proved wrong, the auditor is required to develop additional substantive audit procedures to compensate for the newly determined internal control risk. However, in certain instances, the auditor may conclude that the magnitude of the internal control weakness or the nature of the transactions or accounts being tested is such that it would not be possible to execute sufficient substantive tests to compensate for the control weakness. When this is the case, the auditor will have to issue a qualified opinion or, if warranted, a disclaimer.

HOW TO TEST CONTROLS

Tests of controls must provide assurance on the operational effectiveness of the five control components, as set forth in AU-C 315.A50 (i.e., control environment, risk assessment, information and communication, control activities, and monitoring) in relation to each transaction cycle or accounting application of a government entity.

Audit procedures typically applied in testing the effectiveness of a government’s controls over transactions and accounts include auditor inquiries, auditor-documented examinations, auditor observations, and auditor recalculation of auditee data. Common test procedures include:

• Inquiry: Ask those who do it. Governmental personnel with firsthand knowledge and who are directly responsible for implementing the designed controls must be interviewed. In conducting inquiries, it is important that the auditor interview those with operational knowledge as well as a sound corporate or institutional memory (i.e., an understanding of the entity’s history based on years of experience working for the entity). Auditors consider inquiry to be an essential audit procedure. However, all knowledge acquired through inquiry must be corroborated by other audit procedures. Healthy skepticism is as necessary in the audit of government agencies as it is in the audit of commercial enterprises; reliance cannot be placed solely on inquiry.
• Observation: Witness control-related activities. The auditor might note the performance of certain control-related activities only by observing the actual act by auditee personnel. Examples could include learning of unauthorized personnel who routinely enter restricted computer spaces, managers who consistently delegate check-signing responsibilities to subordinates, or supervisors who habitually sign off on time and attendance records days after payrolls are processed and backdate their actions. Experienced auditors recognize the importance of observation. Even the most well-intentioned employee will occasionally, and inadvertently, develop bad habits, such as failing to secure assets and information, not following restricted access rules, and the like. A competent auditor does not spend all of his or her time at a desk. Auditing by walking around (the combination of observation and inquiry) often discloses important facts and conditions that might otherwise be missed by the execution of more traditional documentation, evaluation, and testing techniques.
• Reperformance: Repeat, recheck, and recalculate auditee work. Sometimes recalculation and rechecking are the primary audit procedures for validating the operational effectiveness of certain control activities. For example, reperformance or recalculation is required by the auditor to validate the propriety of fees billed by a regulatory agency or amounts paid to vendors for goods or services received.
• Documentation: Follow the paper trail. In earlier phases, the auditor should have identified and obtained copies of the various forms that document the control process of each of the government’s transaction groups or accounting applications (e.g., budgeting, tax billings, cash receipts, cash disbursements, payrolls, contract awards, debt issuance). Following the audit trail (often in combination with reperformance and the other techniques just listed) is an essential element in the execution of a sampling plan. Whether derived statistically or judgmentally, the sampling plan applied in the testing of controls requires that the audit trail be examined for a creditable number of transactions within each of the transaction groups or accounting applications. The objective is to confirm that the timing, amount, purpose, quantities of recorded transactions, and appropriateness of accounts charged for the tested transactions are supported by documented authorizations, approvals, and sign-offs.

WALK-THROUGHS AND TESTS OF CONTROL

Determining what to test is one of the key audit decisions related to the testing of controls. At this point, it is important to make a distinction between the documentation developed during the planning phase and concluded during the internal control phase and the controls the auditors decide to test.

In documenting and assessing the system of internal control, the auditor obtains and documents information on all significant activities and procedures and, in his or her evaluation, considers all controls. This is essential to obtaining an understanding of internal controls and the risk assessment process. In obtaining and documenting his or her understanding, the auditor will execute walk-throughs of all, or nearly all, activities and obtain documentation on forms used for inclusion in the audit working papers. During the documentation, the auditor typically examines a small number of transactions to document his or her performance of the walk-throughs.

In contrast, when controls are tested later, audit tests are usually limited to essential or key controls but require the review of a larger number of transactions. This is an important consideration in achieving audit efficiency. All too often, auditors have a tendency to test at the same level of detail as a walk-through, which not only increases their workload but may also detract from their ability to focus on issues of audit importance. How to determine what controls to test is discussed later in this chapter.

TESTS OF CONTROLS ARE NOT EFFECTIVE TESTS FOR TRANSACTION ACCURACY

A recurring misconception, particularly among inexperienced auditors, is that a direct verification of the accounting accuracy of a transaction is a test of controls. During a test of internal controls, the auditor is primarily testing compliance with controls, not verifying the accuracy of the transactions tested.

For example, in a test of controls over cash disbursements, an auditor typically examines documentation such as invoices to test compliance with internal controls. In these cases, the auditor may look for evidence that internal controls were followed in ordering the goods or services (e.g., a purchase order or similar documentation serving as an audit trail), that the invoice was approved for payment by an authorized individual, as required by the system design (audit trail documenting approval), and that the services or goods were received (audit trail documentation, such as a receiving report), among others. Compliance or noncompliance with these processes provides information on the extent to which controls are operational.

In addition, the auditor generally verifies attributes of the transaction itself, such as whether the amount of the disbursement equals the amount on the invoice (or whether a discrepancy is properly explained by the audit trail documentation) and whether the invoice was properly coded. In conducting these tests, the auditor determines whether the amount paid is correct (e.g., agrees with the invoice) and whether it was posted to the proper account (e.g., charged against the correct appropriation, element of expense account, properly capitalized if applicable, etc.). This is an important procedure because the auditor examines just a small fraction of all transactions and cannot afford to overlook any significant aspect of the transaction being tested. However, verifying this attribute is not a test of compliance with internal controls.

Even if the auditor finds no errors affecting accuracy or account classification, he or she cannot conclude by inference that controls promoting the correct payment and classification of expenses are operational. The controls could very well be working, and the accuracy of the transaction is indicative of this, but in these tests, the auditor was not testing any controls. Although reviewing the accuracy of the transaction is important, positive results do not directly support the auditor’s prior assertion regarding internal controls. Conversely, if the auditor determines that an error has occurred, the error cannot be evaluated as another event of noncompliance. Audit judgment must be applied and the prior assessment of control risk (and the audit approach) revisited.

Tests of controls promote audit efficiency in part because the tests recognize that events of noncompliance will occur in even the best of systems supported by a strong control environment. As such, the auditor, using his or her judgment, identifies tolerable maximum noncompliance levels for controls. Thus, for example, the auditor may conclude that the universe of all transactions may include up to a 5 percent rate of noncompliance without forcing the reevaluation of the prior control risk assessment.

However, stating that a 5 percent noncompliance rate is tolerable is not the same as stating that the auditor is willing to accept that 5 percent of the transactions processed by the system are clerically inaccurate, inappropriate/fraudulent, posted to the wrong account, and/or improperly expensed or capitalized. Indeed, if the system produces data that can be up to 5 percent inaccurate, the data produced are very likely to exceed the materiality thresholds established by the auditor during the planning phase.

The reason that a 5 percent noncompliance rate may be tolerable but a similar clerical accuracy error rate is not is due to the objectives and assumptions behind testing internal controls. Stated simply, the fact that there was an event of noncompliance (e.g., no purchase order was issued or one was issued after the fact, the invoice was not matched against the receiving report, etc.) does not necessarily result in an error. Correct transactions can be processed (and often are) by weak systems of internal controls. In addition, in many instances, threats to the processes identified in the auditor’s risk analysis (see discussion in Chapter 10) are totally or partially mitigated by more than one control. To the extent that other compensating controls were operational, the noncompliance event may not be significant. A clerical error or improper payment, however, has different implications, possibly including these:

• Internal controls are perfunctorily executed (e.g., invoices are signed and approved without appropriate review).
• The auditor has failed to identify all threats in his or her risk assessment.
• The auditor’s assessment of the overall internal control environment was incorrect and needs to be downgraded.

At the outset of a test of internal controls, the auditor states what he or she considers to be a tolerable rate of noncompliance. The compliance benchmark may be expressed statistically or judgmentally and may take the form of an absolute number of errors or a percentage rate using statistical techniques. At the end of the test, the auditor compares the actual results against the prior benchmark on compliance. If the benchmark is satisfactorily met (e.g., the error rate is below that which the auditor considered tolerable or acceptable), the auditor may reasonably conclude that the prior assessment of control risk was correct and proceed with the audit approach developed earlier, including, as appropriate, planned substantive testing. If the results are not within tolerable limits, the auditor must revisit his or her approach accordingly. Note that the auditor cannot ignore the implications of a transaction error (e.g., amount, legitimacy, posting) uncovered by a test of controls simply because the test results fall within a tolerable internal control compliance error rate.

CONTROLS TO TEST AND HOW

Factors to consider when deciding what controls to test include those listed next.

• Importance. A key consideration is how critical a control is. In practical terms, the more significant/imminent the threat a control mitigates, the more important the control.
• Number of threats mitigated. Testing a control that prevents or mitigates more than one threat can significantly increase audit efficiency.
• Nature of the threat. Just as one control may mitigate more than one threat, certain threats may be mitigated by more than one control. In general, the auditor may want to avoid testing controls that achieve the same objective. For significant threats, however, the auditor may want to test more than one mitigating control in order to add to his or her audit coverage and/or comfort level.
• Verifiability. In practical terms, to fulfill the auditor’s purpose, a control must be verifiable. Disbursement approval, for example, is more easily verifiable if an audit trail (manual or electronic) is present. Without this trail, sampling is likely to be of little value, and observation may be the only option (but in this case, it may be of limited value to the audit). Faced with no audit trail, the auditor must seek alternative controls supported by appropriate audit trails or may be forced to drop the test of controls and execute extended substantive procedures with account balances.
• Impact of information technology (IT). Advances in IT often result in the loss of a visible audit trail or create other issues that the auditor must consider. For example, the transaction approval process may substitute passwords or encryption for a physical signature. Under these circumstances, the visible audit trail may consist of an indication in the transaction record or database that the transaction was approved at the proper level. Verifying that such a record exists is valid only if accompanied by audit testing procedures that verify controls over password access and encryption.

Selecting the controls to test along with the proper testing technique and approach requires the exercise of judgment and the application of common sense. Common pitfalls include:

• Testing every aspect of the system and giving every control equal significance. All too often, auditors test every aspect of a process, cycle, or system without considering the relationship between a specific procedure and a real threat. Thus, for example, in a poorly thought out test of payroll, the test of control may treat critical controls, such as controls over employee pay changes and adding new employees, as being of equal importance to lesser controls, such as controls over routine payroll forms, such as W-4s, insurance elections, contributions, and so on. (Remember, in the case of Federal audits, noncompliance with some of these lesser concerns may have to be reported as events of noncompliance with laws and regulations.)
• Misinterpreting the purpose of a control. The overabundance of internal control checklists often results in auditors memorizing controls (and incorporating them in their tests) without considering their real purpose. In an example that is more common in the private sector, a typical control is the requirement that disbursements over a certain amount require two signatures. This control is effective in preventing (barring collusion) authorized check signers from unilaterally defrauding their employer of significant amounts of money and is commonly found in most commercial entities. In the absence of check signature stamps (and most medium- and larger-size organizations use signature stamps), the dual-signature requirement can also be considered as a control promoting the use of extra care in approving larger expenditures of funds. However, as is usually the case, when a signature stamp affixes the double signature, the only purpose of the control is the prevention of fraud on the part of the authorized check signer. Further, the existence of a signature stamp raises control issues regarding the proper safeguarding of the stamp. In fact, if the stamp is not properly controlled, the presence of a signature stamp simply transfers the opportunity to commit fraud from a management-level employee to lower-level personnel (or, in the absence of proper access controls, to anyone in the organization). In spite of this discussion, many audit tests include testing for dual signatures while ignoring the security issues and the possibility of exploring a simpler verification procedure. Such a procedure might be confirming directly with a bank that dual signatures are required and that the bank (and not the auditee) is responsible for any potential loss for failing to comply with this requirement.
• Overestimating the control’s positive impact. Time sheets are another example of a control commonly associated with low control risk and often indiscriminately included in a test of controls over payroll. Yet a time sheet that is not subject to external validation, such as a time clock (with controls preventing employees from punching other employees’ time sheets) or the signature of a properly motivated management-level employee cannot effectively ensure that employees are paid only for hours actually worked. Further, in the absence of gatekeeping controls over new employees, the time sheet accomplishes little in preventing the presence of phantom employees.
• Underestimating operating management monitoring controls. Auditors often view sampling (discussed later in this chapter) as the only effective means of obtaining competent evidential matter. This bias is often at the expense of testing other potentially strong controls, such as management’s monitoring of its operations. Without denying or mitigating the critical importance of sampling in auditing, the auditor should be aware of the importance of monitoring controls in reducing control risk and the potential impact the testing of monitoring controls may have on the audit’s efficiency. Monitoring controls also includes observation by both management and executives at all levels of operation as well as by an independent function, such as an internal audit function. While the auditor seldom underestimates the usefulness of an effective, truly independent internal audit function, the same is not the case in connection with operating management’s own monitoring activities. Yet monitoring activities by knowledgeable personnel who are held accountable for results (particularly if the financial records are part of the measuring criteria) significantly impact a system’s ability to produce accurate financial data. It should be easy to see that the presence of operating personnel with a vested interest in how financial results are recorded provides great assurance that financial transactions are charged to the proper appropriation, organization, program, or activity and that all charges are legitimate. Further, monitoring is an activity that can be tested with sufficient objectivity through inquiry, observation, and inspection of records (although, admittedly, certain aspects of the test are, of necessity, more subjective than in audit sampling). Finally, the all-encompassing nature of the Federal budget and the clear priority assigned to it by the Constitution and other laws of the land, politicians, Federal executives, oversight agencies, and the American public ensure the presence (albeit not necessarily the effectiveness) of a monitoring function at all levels of the Federal Government.

Once the auditor decides what controls to test and, ideally, has avoided the potential pitfalls discussed earlier, he or she proceeds with the execution of the different tests of internal controls. Later sections in this chapter discuss the most common form of testing—audit sampling—and other types of tests auditors consider in validating (or disproving) the prior assessment of control risk.

In general, tests of controls provide indirect evidence on the accuracy of account balances by testing whether the systems, processes, and cycles affecting an account are supported by an appropriate internal control design, which operates effectively in accordance with the design. If the controls are appropriate and operating effectively, the auditor concludes that the systems/processes/cycles affecting a particular account can be relied on to produce account balances that are fairly stated in all material respects in accordance with generally accepted accounting principles (GAAP).

SUBSTANTIVE TESTS

Substantive tests typically are performed in combination with tests of controls to provide additional support for specific audit assertions. Although auditors are allowed to perform substantive tests only, they still must obtain the necessary evidence to ensure that they have properly understood the system of internal controls and that the execution of substantive procedures decreases the audit detection risk to an acceptable level.

Substantive testing provides direct evidence on an account balance regardless of the systems and/or controls interfacing with the account. Substantive testing may be defined as audit procedures designed to detect material misstatements at the assertion level. The AICPA has classified substantive audit tests into two groups (AU-C 330.04):

1. Test of details. Classes of transactions, account balances, and disclosures

2. Analytical procedures. Evaluation of financial information through analysis of plausible relationships among both financial and nonfinancial data

Substantive testing does not ignore the results of tests of internal controls. In fact, the extent and nature of substantive testing is usually dictated by the auditor’s internal control evaluations and, when applicable, testing results.

The extent of substantive testing will, in most instances, be determined by these factors (or combinations thereof):

• The auditor’s evaluation of internal controls. In general, the extent of testing is directly related to the auditor’s final determination of audit risk on completion of audit testing (if audit testing was deemed appropriate). The need for substantive testing decreases as the auditor’s confidence with and reliance on internal control increases.
• The specific weaknesses in internal controls identified during evaluation and/or testing.
• Audit efficiency when the auditor determines that substantive tests will require a lower level of effort than a test of internal controls, regardless of the auditor’s risk assessment (but see the next requirement).
• The OMB requirement that internal controls be tested when the auditor concludes that an effective internal control design is in place. If the auditor must comply with this OMB requirement even though a substantive test may be more efficient, he or she usually will develop a hybrid test combining substantive procedures with internal control testing.

AICPA guidance (AU-C 330.18) actually requires the execution of substantive procedures “for all relevant assertions related to each material class of transactions, account balance, and disclosure.” The guidance goes on to explain that this is a necessary check on the judgmental nature of the auditor’s evaluation of internal control.

In addition, AU-C 330.21 requires these specific substantive audit procedures:

• Agreeing or reconciling the financials and notes to the underlying accounting records (e.g., tie balances to the GL).
• Examining material journal entries and adjustments made during the financial statement preparation process.

Readers may note the use of the word should as opposed to must in regard to these requirements. We again suggest that most, if not all, shoulds be treated as musts. If nothing else, the requirements just listed are simply commonsense ones.

Validating Account Balances

Auditing the existence of account balances can be accomplished directly by examining the documentation or other physical evidence substantiating the validity of the GL balance. Typical procedures are listed next.

• Testing a sample of transactions posted to the account throughout the year or period under audit. (Audit sampling is discussed later in this chapter.) This typically includes the examination of documents supporting the transaction (e.g., invoices). As a general rule, the auditor:
  • Concentrates on large transactions to, in effect, “audit” a sufficient percentage of the total account balance.
  • Reviews the file for unusual transactions.
  • Develops projections from sampling results usually applying statistical techniques.
  • Employs a combination of these procedures.
• Physically inspecting the account components (e.g., physical inventory observations; verification of the physical existence of property, such as buildings).
• Directly confirming account components (usually in connection with accounts receivable but may also be used in connection with inventories held by a third party and with accounts payable).
• Reviewing and verifying account reconciliations including reconciliation of:
  • Fund balance with Treasury (or cash held outside Treasury).
  • Detailed subsidiary ledgers (e.g., accounts receivable or payables) to the GL’s control account.
  • Property records kept by a custodian function (e.g., perpetual inventory records and property, plant, and equipment records) to the GL control account.
  • Physical inventory counts to the GL control account.
• Recomputing auditee estimates and other auditee-developed data to verify accuracy of estimate or computation (e.g., footing auditee prepared analysis, recomputing auditee’s accruals).

Analytical Procedures

In general, analytical procedures compare different but related sets of financial data to assess whether the relationships and variances between the data are consistent with the auditor’s expectations. Typical relationships that auditors consider are listed next.

• Comparing prior-year account balances to current-year balances. In this procedure, the auditor looks for unusual variances in account balances from one year to the next. Unusual variances generally consist of significant increases or decreases that cannot be explained by other known variables (e.g., increases or decreases in programs and activities, new programs, program terminations, etc.). However, the auditor must consider that the lack of an increase or decrease in an account may also constitute an unusual variance. This would be the case, for example, if a program is terminated but accounts related to the program remain unchanged from the prior year.
• Considering the consistency of financial ratios. In this procedure, the auditor considers whether relationships, such as the relationship between the agency’s budgetary and proprietary accounts, revenues to accounts receivable, expenses to accounts payable, and so on, are consistent with the entity’s historical ratios. This is an important procedure because, for example, a decrease in the ratio of revenues to accounts receivable may be indicative of a deteriorating receivable. Similarly, if a working capital fund shows a decrease in the ratio of sales to inventory (inventory turn ratio), this condition may be indicative of obsolete or damaged inventory.
• Performing rough tests of account balances. Under this procedure, the auditor develops a simple estimate of what an account value should be and compares this value with the actual account balance. A typical example consists of developing an estimate of accrued salaries based on the last pay period of the year and comparing this estimate to the agency’s recorded accrued salaries.
• Projecting/estimating account balances. This procedure is similar to developing rough tests of account balances (just mentioned). However, projections and estimates are usually more complex and often make use of statistical procedures, including regression analysis, to project/estimate account balances. Typical applications include:
  • Estimates for uncollectible receivables. This may include, for example, considering the aging of accounts receivable and developing a relationship (statistical or judgmental) between the aging and future write-offs to generate an estimate of uncollectible receivables.
  • Grant accrual estimates. A significant number of Federal Agencies are involved in the issuance of grants to individuals, institutions, and state and local agencies. For these agencies, the year-end grant accrual is material to the agency’s financial statements. Given the current tight deadlines for the issuance of Federal financial statements, auditors cannot rely on more traditional procedures (e.g., review of grant activity/payments after year-end) and must rely on projections (statistical or judgmental) to validate the reasonableness of the grant accrual.
  • Year-end liability estimates. Before closing its books, each Federal Agency must consider outstanding debts and develop a reasonable estimate of the liability for services or goods received that have not been invoiced by vendors (or that have not yet been entered into the appropriate accounting system). In the past, the traditional audit procedure consisted of performing a test for unrecorded liabilities where the auditor examined postclosing disbursement/invoice vouchering activity for a number of months subsequent to year-end to test the reasonableness of the year-end estimate. As was the case with grants, the shortened reporting deadlines typically require the execution of analytical procedures to test/validate the reasonableness of the year-end balance for accounts payable.

Analytical procedures are not limited to the testing phase. In fact, as noted, AICPA guidance (AU-C 315.06) requires that the auditor use analytical procedures in planning and performing risk assessment procedures. It should be clear that procedures such as comparing balances from year to year and assessing the implications of financial ratios would quickly identify areas of potential audit significance that should be considered during the planning phase and in the development/revision of audit programs.

The execution of analytical procedures in connection with substantive procedures is left to the auditor’s judgment. However, analytical procedures are often nonlaborious, time-saving procedures that can significantly increase the auditor’s efficiency. Moreover, it should be apparent that the complexity and size of Federal Agencies, coupled with ever-tightening reporting deadlines, leave the auditor with no option but to make use of these techniques. AICPA guidance requires that when the analytical procedure is a significant substantive test, the expectations, results, and additional related procedures performed be documented.

Related procedures usually include inquiries of the auditee (e.g., to explain unusual variances) and limited tests of accounting data. It is important that the auditor independently test information provided by the auditee and that he or she ensures the integrity and completeness of the universe from which accounting data are selected for testing. (The importance of ensuring data integrity and completeness is discussed in more detail later in this chapter.)

AUDIT SAMPLING

Sampling is a common audit procedure. The AICPA defines sampling as the execution of audit procedures to less than 100 percent of the items in the universe or account being audited (AU-C 530). At the outset, it should be understood that sampling encompasses judgmental, nonrandom procedures as well as statistical sampling. In practice, the auditor often combines judgment and statistics in the execution of sampling-related procedures. This section discusses sampling as it relates to both tests of controls and substantive testing, goes over some dos and don’ts in the use of sampling, and concludes with a summary discussion of statistical sampling.

Sampling Applications

Sampling can be used in connection with the two types of audit tests discussed earlier. Sampling in connection with tests of controls and substantive tests differ in terms of the information that the auditor examines in the execution of his or her test.

Controls Testing

In the execution of tests of controls, the auditor looks for evidence that a control was complied with. For example, the auditor looks at supporting documentation, such as receiving reports, invoices, purchase orders, and management approvals, to verify that goods and services were ordered and approved in accordance with the procedures and controls that were previously considered in the determination of control risk. Similarly, the auditor may look for approved time sheets and approved payroll registers (or evidence that the register was reviewed by management) to test key controls identified in the evaluation of the payroll cycle.

Substantive Testing

In the execution of substantive procedures, the auditor is concerned with the accuracy of the account balance, not with the controls present in the processes/cycles that created the account balance. Again, the auditor looks at supporting documentation. However, in substantive testing, he or she is interested in establishing the accuracy of the transaction recorded in the account. Thus, the auditor may look at a vendor invoice to ensure that the account is properly valued or at a sales invoice to support an individual balance in accounts receivable.

SAMPLING CONSIDERATIONS

This section expands on some key issues that the auditor should consider when using sampling techniques.

Ensure Completeness of the Universe

The purpose of the sample is to make an assertion on the population being tested, such as whether internal controls are operational (in a test of controls) or whether account balances are fairly stated (in a substantive test). These assertions are the result of the auditor’s examination of a (usually) small percentage of the population. Therefore, it is essential that the auditor ensure that the universe being audited includes all the transactions processed by the cycle or affecting an account balance for the period covered by the test.

Prior to selecting a sample, the auditor obtains a record or database of all appropriate elements of the universe to be sampled. In general, the universe consists of:

• Test of controls. In the test of controls, the universe under audit typically includes all transactions processed by the cycle or system of internal controls, such as all cash disbursements, all cash receipts, all payroll checks, and so on, for the period tested.
• Substantive test. In a substantive test of an account balance, the universe is usually defined in terms of specific items, such as individual accounts receivable or payable, properties, inventory items, and the like that make up the account balance at the end of the year (or interim period under audit). The universe can also be defined in terms of all transactions that affected the account during the period under audit (e.g., beginning balance plus all debits and credits affecting the account). While this latter approach may be appropriate under certain circumstances (e.g., in the audit of property, plant, and equipment where the beginning balance was previously audited), defining the universe in terms of the specific items making up the account balance is often preferable (e.g., list of vendor amounts included in an accounts payable balance).

The auditor is ultimately interested in being able to issue an opinion on the financial statements. The financial statements are derived from the books and records of the auditee (typically, the GL plus year-end adjustments). Therefore, to ensure the completeness of the universe, the auditor must ensure that the universe was, in fact, derived from the GL (including adjustments, if applicable).

In the case of an account balance audit, this procedure is relatively simple. The auditor adds the schedule of all items making up the balance (e.g., all accounts receivable) and verifies that the sum of these items agrees with the GL balance (including year-end adjustments, if applicable).

Under certain circumstances, reconciling the database to the accounting records in a test of controls may be rather straightforward. For example, in the case of a payroll test, the auditor may be able to exactly match the total of the universe tested to a payroll expense account in the GL. Similarly, if the audit consists of testing benefit payments (e.g., in connection with the audit of an insurance or social benefit fund), the auditor may be able to relate the sum of these payments to a benefit expense account.

In practical terms, however, validating the universe in a test of internal controls is not a straightforward procedure. Even in the simplified examples given, the auditor is likely to encounter difficulties in reconciling his or her database to the accounting records since it is almost certain that the payroll expense account and the benefit expense account, as well as the cash accounts, will include other transactions (e.g., journal entries, adjustments, accruals, etc.). Although validating the universe is not always a simple procedure, if the auditor has obtained a sound understanding of the accounting cycles and processes during the planning and internal control phases, he or she will be able to efficiently relate the universe being tested to the books of original entry.

The failure to validate the completeness of the universe being tested is a recurring problem with the quality of Federal audits. Indeed, the authors of this book, on more than one occasion, have encountered situations where large samples were selected to test a critical process but the database from which the sample was selected was never agreed to the accounting records. Moreover, some of the recent auditing failures can be related, at least in part, to a failure to ensure the completeness of the universe. Selecting a sample without validating the universe from which it is selected renders the audit test meaningless.

Analyze the Universe

IT provides the auditor with ample opportunity to increase audit efficiency and ensure that the audit focuses on areas of potential audit significance. In most Federal audits, the universe from which the sample is to be selected consists of an electronic file or database, ideally including all of the elements to be tested.

Electronic files provide the auditor with the opportunity to analyze the characteristics of the universe and, in certain cases, even “audit” 100 percent of certain attributes. Typical procedures that can be performed to increase audit efficiency are listed next.

• Obtain a frequency distribution of the file. In testing disbursements of any kind (e.g., benefits, grants, payroll, vendors), a frequency distribution provides numerous opportunities to improve audit efficiency.
  • Identify large payments. This procedure allows the auditor to concentrate on large payments (thus increasing audit coverage) and/or to identify unusual payments for follow-up.
  • Increase the efficiency of statistical applications. Stratification of the universe often increases audit efficiency by reducing the impact of the variability of the population on the sample size.
  • Support the execution of analytical procedures. For example, comparing the current year’s frequency distribution to that of a prior year’s may uncover unusual trends that could merit audit consideration.
• Execute file matches. It may be possible to test 100 percent of attributes by executing file matches. For example:
  • In a test of benefit payments, it is possible to match payments to beneficiaries with the Social Security Administration’s files to ensure that benefit checks do not continue to be issued after a beneficiary’s death.
  • A match of this year’s file to the prior year’s can identify new payees and enable auditors to test gatekeeping controls.
• Support exception testing. The auditor may be interested in testing payments based on certain attributes in the file. In addition to large payments, the auditor may be looking for other characteristics, including, for example, all disbursements:
  • Charged to a specific element of expense account (e.g., all payments charged to a maintenance account or property, plant, and equipment to test for incorrect classifications).
  • To a particular payee (e.g., to test for reimbursable travel expenses and/or advances to executives and other members of management).

In summary, databases and files provide the auditor with significant sampling opportunities to increase audit efficiency and properly focus the audit effort.

Relationship of Control Testing to Substantive Testing

An effective audit approach relates substantive testing to internal control testing. Substantive testing cannot take place in a vacuum. Substantive testing is derived entirely from the auditor’s evaluation of internal control. Auditors perform substantive testing for a variety of reasons, but the extent of substantive tests is always dependent on the state of an entity’s internal controls. Weak or ineffective controls all result in the need to do more extensive and possibly different substantive tests, including a controls environment that:

• Lacks effective internal controls (either due to flaws in the design or to noncompliance, as noted during internal control testing).
• Requires that the auditor compensate for certain weaknesses in internal controls.
• Mandates the auditor provide additional audit coverage on the large account balances.

Notwithstanding the design of an efficient audit approach and lack of effective controls, for all significant systems, the auditor must perform sufficient control tests to comply with OMB’s Federal audit guidance.

Assuming that the auditor properly performed the risk assessment during the prior phase of the audit, the verification procedures encompassed by substantive testing will ensure that the audit effort focuses on aspects of the account balance that are at risk of being misstated.

It would be erroneous to conclude that substantive testing results never have an impact on the auditor’s evaluation of internal control. As noted earlier, planning and reevaluating the audit approach are never-ending aspects of every audit. Thus, substantive testing often provides additional information on internal controls that the auditor must take into account. Results and related actions that the auditor considers include:

• Adverse results. The execution of substantive procedures may disclose significant errors that will require audit adjustments. Depending on the prior assessment of control risk as high, moderate, or low (following Financial Audit Manual [FAM] terminology), the auditor will:
  • If control risk was evaluated as high, consider whether the substantive procedures in the audit approach are sufficient to ensure that all material errors will be identified and corrected. Careful audit judgment must be exercised to ensure that conditions anticipated during the earlier evaluation are no worse than expected. If the auditor’s worst expectations are exceeded, he or she must reconsider whether substantive procedures can overcome the risk that accounts are not fairly stated. Further, it is important to note that there are cases where the exclusive use of substantive procedures will not provide sufficient audit evidence to support the auditor’s assertions.
  • If control risk was evaluated as moderate or low, consider whether errors disclosed by the procedures force the auditor to revise his or her evaluation of internal controls. In turn, this reevaluation often results in additional substantive testing or, if the results are adverse enough, a reconsideration of whether substantive procedures can overcome the risk that accounts are not fairly stated.
• Positive results. It is possible that substantive procedures disclose fewer errors than anticipated (or even no errors) based on the auditor’s risk assessment. The auditor could, of course, ignore the positive results without increasing his or her audit risk (e.g., conclude that the account is fairly stated when, in fact, it is not). However, a careful evaluation of the results may lead the auditor to discover the presence or effectiveness of a control that was missed or not properly considered in the evaluation of internal controls. If this were the case, the auditor would be in a position to revise his or her audit approach. Although it is unlikely that this finding will increase the efficiency of the current year’s audit, it may favorably impact the auditor’s efficiency in future audits. Additionally, it may impact the nature and extent of required communications with management and/or those in governance.

Statistical Sampling

Statistical techniques and statistical sampling are usually essential to the execution of an efficient audit strategy. Statistical sampling has the advantage of producing measurable results and, in many cases, achieves audit objectives with a relatively small sample size. This section summarizes some key aspects of statistical sampling in auditing, but a full discussion of statistical sampling is beyond the scope of this book. There are a number of very helpful documents in this area that the reader may wish to consult, particularly:

• GAO’s Using Statistical Sampling (May 1992 revision). A very helpful how-to document with insightful discussions on attribute sampling and variables estimation procedures
• AICPA’s audit guide titled Audit Sampling (June 22, 2012 edition). Provides helpful guidance on statistical sampling including a very valuable discussion on the use of probability proportional to size sampling

The successful application of statistical sampling in an audit requires the careful identification, definition, and evaluation of a number of variables. A discussion of some of these variables follows.

• Population. The universe of items from which a sample is taken (e.g., all disbursements, all receipts, all payroll payments).
• Sampling unit. Any of the individual items encompassed by the population (e.g., a check, a deposit, an invoice). In defining the sampling unit, the auditor must consider the attributes to be tested or account balance to be estimated as well as the ease with which a sample can be selected.
• Error. In attribute testing, an event of noncompliance with internal controls and/or clerical errors uncovered by testing procedures.
• Sampling risk. Relates to the possibility that the test includes a sample that will lead the auditor to reach an incorrect decision regarding the characteristics of the universe. In statistical sampling, risk can be expressed as incorrectly concluding that the controls tested are:
  • Being followed or complied with or that the account balance is not materially misstated. This risk may result from the auditor issuing an unqualified opinion when, in fact, such an opinion is not warranted.
  • Not being followed or complied with or that the account balance is materially misstated. In internal control testing, this may lead the auditor to perform additional substantive procedures, thus needlessly decreasing audit efficiency but not affecting the audit assertion (e.g., the auditor is guilty of overauditing but not of reaching the wrong conclusion and/or type of opinion). However, in substantive testing, this type of error may lead the auditor to propose an audit adjustment that materially misstates the account balance under audit.

Statistical sampling supports both test of controls and substantive tests. Types of tests or approaches typically used in auditing include:

• Attribute sampling. Attribute sampling is most commonly used in tests of controls and tests of compliance with laws and regulations. Under attribute sampling, each unit tested will have one of two mutually exclusive characteristics (e.g., the control was complied with, or the control was not complied with). This procedure allows the auditor to develop an estimate of the universe’s error rate (e.g., the extent to which a control is not being complied with expressed as a percentage of the total population). In this type of test, the auditor is concerned with the maximum error rate of the population (at a given confidence level). If the maximum error rate is within a predetermined compliance range (e.g., no more than 5 percent), the auditor generally concludes that controls are effective. If the error rate is higher than the tolerable limit, then the auditor revises the risk assessment and modifies the audit approach, as necessary.
• Variable estimation sampling. This sampling develops a projection within which the true audited value of the population lies. In this type of test, the auditor determines a true or audited value for an item in the population (e.g., an invoice or a check) and from the audited value generates an estimate of the total population. The auditor compares the estimate with the actual account balance, and, assuming the desired sampling precision was met, the auditor either concludes that the account balance is fairly stated or proposes an audit adjustment to correct the balance. Audited value relates to the correct amount of the sample item (e.g., a recorded invoice, check, individual account receivable or payable, etc.) regardless of the value at which it was recorded by the accounting cycle. Thus, the audited value can differ from the value recorded in the books and records of the auditee (i.e., because the transaction was incorrectly recorded).

Variable estimation sampling encompasses direct, difference, and ratio estimations. Direct estimation consists of developing a projection of an account balance by utilizing the true or audited value of the sample items (e.g., an estimate of total accounts receivable from the audited value of all accounts receivable selected in the sample). A major problem with direct estimation is that obtaining the required precision for the estimate to be within tolerable limits often requires very large sample sizes. This problem can be overcome by the application of difference and/or ratio estimation techniques. Difference estimation consists of developing an estimate of the account balance by taking into account the difference between audited and recorded or book value. Ratio estimation develops the estimate based on the ratio of the audited value to the recorded value. In both cases, the approach is likely to significantly reduce the size of the sample required to develop projections that fall within tolerable limits.

Although difference and ratio estimation techniques can significantly increase the precision of the estimate, they should not be used unless a minimum number of errors are included in the randomly selected sample. (GAO recommends a minimum of 10 errors and notes that some statisticians believe the number should be as high as 30 errors.)

• Probability proportional to size sampling (PPS). This method combines characteristics of both attribute sampling and variable sampling. This approach is also known as dollar unit sampling because in PPS, the sampling unit is each dollar included in the population or account. That is, each dollar in an account has an equal chance of being included in the sample (thus providing higher-value items/transactions with a proportionately greater chance of being selected). PPS allows the auditor to combine an attribute sampling approach with the ability to develop dollar projections. This approach is more informative than attribute sampling because it expresses results in terms of dollars. In addition, PPS is easier to apply than variable estimation sampling and typically requires a smaller sample size. As a result, PPS is, at least arguably, rapidly becoming the most often used statistical technique in auditing.

INFORMATION TECHNOLOGY CONSIDERATIONS

IT alters the fundamental manner by which transactional data are initiated, input, recorded, compiled, classified, and ultimately reported. IT-driven systems paper trails common to most manual systems disappear to support enhanced economy and efficiency, the duties and responsibilities of computer-based systems personnel are different, and maximum segregation of duties of the old systems may no longer exist or be relevant. It is important to heed the AICPA’s guidance with respect to technology and computerized data processing systems.

• IT provides the potential benefits of effectiveness and efficiency for an entity’s internal control because the technology enables an entity to:
  • Consistently apply predefined business rules and perform complex calculations in processing large volumes of transactions or data.
  • Enhance the timeliness, availability, and accuracy of information.
  • Facilitate the additional analysis of information.
  • Enhance the ability to monitor the performance of the entity’s activities and its policies and procedures.
  • Reduce the risk that controls will be circumvented.
  • Enhance the ability to achieve effective segregation of duties by implementing security controls in applications, databases, and operating systems (AU-C 315.A56).
• IT also poses specific risks to an entity’s internal controls, including:
  • Reliance on systems or programs that are processing data inaccurately, processing inaccurate data, or both.
  • Unauthorized access to data that may result in the destruction of data, or improper changes to data, including the recording of unauthorized or nonexistent transactions or inaccurate recording of transactions.
  • Unauthorized changes to data in master files.
  • Unauthorized changes to systems or programs.
  • Failure to make necessary changes to systems or programs.
  • Inappropriate manual intervention.
  • Potential loss of data (AU-C 315.A57).
• The extent and nature of these internal control risks vary depending on the characteristics of the entity’s information systems. For example, multiple users, either external or internal, may access a common database of information that affects financial reporting. In such circumstances, a lack of control at a single user entry point might compromise the security of the entire database, potentially resulting in improper changes to or destruction of data. When IT personnel or users are given, or can gain, access privileges beyond those necessary to perform their assigned duties, a breakdown in segregation of duties can occur. This could result in unauthorized transactions or changes to programs or data that affect the financial statements. Therefore, the nature and characteristics of an entity’s use of IT in its information system affect the entity’s internal control (AU-C 315.A60).

OUTSOURCING ACCOUNTING AND DATA SERVICES

Outsourcing of data services does not eliminate but rather aggravates the data control risks by fundamentally altering the approach to testing and validating the data processes of an agency.

Whether data are processed, accounted for, and reported by a manual system, an in-house computer processing facility, or outsourced in whole or in part to an external provider of these services, the fundamental control issues and the responsibilities of the auditor do not change. Under each scenario, the auditor is required to gain an understanding of each of the components of an agency’s internal control structure (i.e., the controls environment; the agency’s regular risk assessments; the control activities, policies, and procedures for implementing management directives; the supportive information and communication systems for identifying, capturing, and reporting; and the monitoring process to assess the quality of controls).

When data services have been outsourced, the servicing entity could be responsible, under the outsourcing contract, for part or all of the original systems design and software as well as data input, processing, and reporting. Under these conditions, the outsourcing Federal Agency has limited or no management control over the data being processed/generated, yet the quality of services provided by the servicing entity become controls and systems considerations for the auditor. The AICPA suggests that information concerning the servicing organization controls and systems be examined, tested, and validated from a wide variety of sources, including:

• The servicing organization’s user manuals.
• Systems documentation, technical manuals.
• The terms, conditions, and scope of services of the outsourcing contract.
• Reports issued by the servicing organization’s independent auditor, internal auditor, and retained consultants.
• Reports of regulatory agencies.

A specific section of the AICPA’s Professional Standards (AU-C 402, Service Organizations) provides considerable guidance on addressing the control risks associated with data processing, accounting, and reporting that have been outsourced to an organization external to the audited agency. Over the years, an increased use of service organizations has caused the AICPA to issue several related statements on auditing standards (SAS). The latest such issuance currently in effect is Statements on Standards for Attestation Engagements (SSAE) 16, Reporting on Controls at a Service Organization.