Evaluating IT General Control Deficiencies

To assess IT general controls at the application or transaction level, internal auditors can use the "Framework for Evaluating Control Exceptions and Deficiencies," which helps to identify control deficiencies, significant deficiencies, and material weaknesses during Section 404 compliance audits.

Section 404 of the U.S Sarbanes-Oxley Act of 2002 requires publicly traded companies to provide an annual attestation of their internal controls over financial reporting. During these control examinations - which are performed by management or a company's external auditors - internal control problems are evaluated and categorized as control deficiencies, significant deficiencies, or material weaknesses. To evaluate a company's internal controls over financial reporting, internal control processes, or transaction-related controls, IT general controls (ITGCs) need to be considered, which can impact the security of financial applications and data.

In 2004, nine public companies developed a methodology for evaluating ITGC deficiencies. Known as "A Framework for Evaluating Control Exceptions and Deficiencies" (the Framework), this methodology can help companies assess the effectiveness of ITGCs and detect the presence of errors in financial statements. More specifically, the Framework can be used when evaluating control exceptions and ITGCs with and without application-level deficiencies.

Before Using the Framework To ascertain the impact of ineffective ITGCs on internal controls over financial reporting, organizations first need to identify or scope associated control objectives and risks. Proper scoping helps organizations to determine which ITGCs need to be assessed as part of the audit, as well as identifies complementary or compensating controls in the event ITGCs do not operate effectively. The Institute of Internal Auditors (The IIA) developed a scoping methodology called Guide to the Assessment of IT General Controls Scope based on Risk (GAIT). This methodology enables internal auditors to assist management and external auditors determine the extent of in-scope ITGCs that are subject to Section 404 compliance. Once the appropriate controls are scoped, companies can use the Framework or other evaluation criteria to identify any internal control failures or deficiencies.

GETTING STARTED

The Framework consists of four charts that provide a systematic approach for evaluating control exceptions and deficiencies. Chart 1 requires the auditor to determine whether control objectives were met and if an identified exception gives rise to a control deficiency. Chart 2 uses quantitative and qualitative criteria to determine if an identified deficiency is a control deficiency, significant deficiency, or material weakness. Chart 3, which also requires an analysis of quantitative and qualitative criteria, is then used to evaluate ITGCs with and without application-level deficiencies. Finally, Chart 4 is used to evaluate pervasive controls other than ITGCs. Typically, environment-type controls are evaluated in Chart 4.

Evaluating Control Exceptions
After ITGCs are scoped properly, internal auditors need to identify the appropriate control deficiency category as well as any control exceptions. A control exception exists when test evidence indicates that the affected control has not been performed as documented. If an exception is identified, Chart 1 is used to determine whether the exception is at least a control deficiency. For example, assume that an internal auditor tests a daily network intrusion report and finds that one report out of a sample of 30 was not reviewed. Because the internal auditor expects all 30 reports to show evidence of a review, an evaluation of this exception using the Framework would look like this:

Evaluated ITGC Control: Network 1 Description: Daily, intrusion detection reports are reviewed by network personnel. Corrective action is taken and documented. Objective: To monitor for potential malicious or unauthorized entry into the corporate network. Risk: Malicious or unauthorized entry into the corporate network. Chart 1: Box 1 Requirement: Examine and understand the cause and results of exceptions. Was the test objective met? Answer: No Notes: Reviewer and back up did not review a daily report.

As shown above, the answer to Box 1 of Chart 1 is "No" because the internal auditor found an instance in which the control was not performed. Even though one exception exists, the internal auditor may choose to pull another sample to determine whether the one exception is representative of the population. Assuming that the internal auditor selects another sample of 30 reports and finds no exceptions within that sample, Chart 1 would look like this:

 

Evaluated ITGC Control: Network 1 Description: Daily, intrusion detection reports are reviewed by network personnel. Corrective action is taken and documented. Objective: To monitor for potential malicious or unauthorized entry into the corporate network. Risk: Malicious or unauthorized entry into the corporate network. Chart 1: Box 1 Requirement: Examine and understand cause and results of exceptions. Was the test objective met? Answer: No Notes: Reviewer and backup did not review daily report. Chart 1: Box 2 Requirement: Could additional testing support a conclusion that the deviation rate or observed exception is not representative of the total population? Answer: Yes Notes: This is a daily control and adequate sample size exists. If any of the answers to Boxes in 1 - 2 is "Yes," then go to box 3. If the answer is "No," a control deficiency exists; go to Chart 3. Chart 1: Box 3 Requirement: Extend testing and re-evaluate. Was the test objective met? Answer: Yes Notes: Retested another 30 reports and found no exceptions. If the answer to Box 3 is "Yes," then it is not a control deficiency. If the answer is "No," a control deficiency exists; go to Chart 3. Conclusion for Chart 1: Not a control deficiency

The answer to Box 2 of Chart 1 is "Yes" because the internal auditor believes that another sample of 30 would yield no exceptions. The auditor may reach this conclusion based on the frequency of the control, the personnel performing the control, prior history of the control operation, and other factors. The answer to Box 3 of Chart 2 is also "Yes" because the one exception found is now in relation to 60 samples instead of 30. The internal auditor may conclude that the one exception is not representative of the population and the overall objective of the control was met.

Unfortunately, not all exceptions are categorized under the "not-a-control-deficiency" category. If the internal auditor is testing quarterly reviews of server security configurations and one of the two quarters examined shows that a review was not performed, Chart 1 would look like this:

 

Evaluated ITGC Control: Server 1 Description: Quarterly, server configurations are reviewed by network personnel to ensure that settings are in accordance with corporate policy. Objective: To determine whether server security is established and maintained by the appropriate settings. Risk: Inappropriate server configurations could compromise security. Chart 1: Box 1 Requirement: Examine and understand the cause and results of exceptions. Was the test objective met? Answer: No Notes: 1 of 2 quarterly reviews of server configurations was not performed. Chart 1: Box 2 Requirement: Could additional testing support a conclusion that the deviation rate or observed exception is not representative of the total population? If any of the answers to Boxes 1 - 2 is "Yes," then go to box 3; If the answer is "No," a control deficiency exists; go to Chart 3. Chart 1: Box 3 Requirement: Extend testing and re-evaluate. Was the test objective met? If the answer to Box 3 is "Yes," then it is not a control deficiency. If the answer is "No," a control deficiency exists; go to Chart 3. Conclusion for Chart 1: Control deficiency

Since a quarterly control has one exception out of a population of two, the internal auditor could not argue that the exception is not representative of the population. To determine the overall categorization of this deficiency, an analysis using Chart 3 will be necessary.

Reviewing ITGCs Without Application-level Deficiencies
The Framework can be used to identify control deficiencies at the application level by using Chart 3. Analyzing deficiencies with Chart 3 requires an internal auditor to judge the deficiency from both an IT perspective and a business or transaction process perspective. For instance, some ITGCs are less closely related to the financial statement than others. A good example is network controls, which are farther away from the financial statements than those associated with the applications and databases that generate the financial statements. The rationale behind this methodology is that ITGCs are pervasive throughout any organization and the failure of ITGCs could trickle down into transactional processing.

Continuing with the example on the quarterly server configuration review, let's assume that no other controls related to server security configurations were tested. Additionally, let's assume that the applications and databases located within the affected server have effective security controls and all other application and database controls are effective. Based on these facts, an analysis under Chart 3 would look something like this:

 

Evaluated ITGC Control: Server 1 Chart 3: Box 1 Requirement: Are there complementary or redundant ITGCs that were tested and evaluated that achieve the same control objective? Answer: No Notes: The review of quarterly server configurations is a detective control and no other related controls exist. If the answer to Box 1 is "Yes," then go to Box 5. If the answer is "No," go to Box 2. Chart 3: Box 2 Requirement: Are there control deficiencies at the application level evaluated in Chart 2 that are related to or caused by the ITGC deficiency? Answer: No Notes: No application level deficiencies exist that stem from the failed server control. Additionally, application security controls are effective. If the answer to Box 2 is "Yes," then go to Box 3; If the answer is "No," go to Box 5. Chart 3: Box 3 Requirement: Are there control deficiencies at the application level related to or caused by the ITGC deficiencies that are classified as only a deficiency? If the answer to Box 3 is "Yes," then go to Box 5; If the answer is "No," go to Box 4. Chart 3: Box 4 Requirement: Are the control deficiencies at the application level related to or caused by the ITGC deficiency that are classified as a significant deficiency? If the answer to Box 4 is "Yes," then a significant deficiency exists - end of test. If the answer is "No," a material weakness exists - end of test. Chart 3: Box 5 Requirement: Does additional evaluation result in a judgment that the ITGC deficiency is a significant deficiency or would a prudent official conclude that the ITGC deficiency is a significant deficiency? Answer: No Notes 1. A quarterly review was not performed for one of the in-scope servers. 2. This deficiency only impacts one in-scope server which houses two in-scope applications. 3. The company's systems environment is relatively simple. 4. The deficiency could impact access to two applications; however, any application access errors would have been caught by application access reviews. 5. Cause of the deficiency was due to human oversight. 6. No history of restatement exists due to this error. If the answer to Box 5 is "Yes," a significant deficiency exists - end of test. If the answer is "No," a control deficiency exists - end of test. Conclusion for Chart 3: Control deficiency Overall Conclusion: Control deficiency

Box 1 of Chart 3 helps the auditor identify if complementary or redundant controls exist that offset the deficient control. A complementary control is one that operates at the same level of precision as the deficient control, while a compensating control is one that operates at a higher level of precision than the deficient control. The control's level of precision impacts the degree of a potential misstatement that could arise due to a deficient control. Therefore, if a complementary control exists for the deficient control, and the control was shown to be effective, then the degree of potential misstatement is highly mitigated. On the other hand, if only compensating controls exist, then the degree of potential misstatement is mitigated to the extent that the compensating control "catches" errors or misstatements. In the example above, no other controls exist that would detect or prevent inappropriate server security configurations.

Box 2 of Chart 3 bridges the gap between the ITGC and its potential impact to transaction process controls. Box 2 also asks the internal auditor to identify whether deficiencies at the application level exist because of the deficient ITGC. Because ITGCs can impact an organization's financial applications and related data, Box 2 asks whether a deficiency at the ITGC level could lead to other problems in the organization.

Box 5 of Chart 3 requires the auditor to employ sound reasoning and objectivity because of the subjective nature of the qualitative measure. Below are some of the subjective questions that Box 5 requires the internal auditor to evaluate:

• Nature and significance of the deficiency. The root cause and timing of the deficiency provides insight and a better understanding of the facts and circumstances surrounding the control deficiency.
• The pervasiveness of the deficiency. A deficiency that is isolated to a specific application or data set poses less risk of misstatement than one that impacts several applications or data sets.
• The complexity of the company's environment. In general, controls related to routine processes or those that require little subjective judgment are associated with processes that do not carry a high degree of misstatement risk. In contrast, controls that are non-routine or require a high degree of subjectivity are susceptible to a higher degree of misstatement.
• The relative proximity of the deficient control to applications or data. Deficiencies at the application or data level pose higher risks of misstatement than those at the network or operating system level. Additionally, the reliance on transaction-based controls as compensating controls becomes more important when deficiencies at the application or data level exist because compensating controls at the ITGC level may not adequately mitigate the risk of misstatement.
• Cause and frequency of the deficiency. Human-type errors could be construed as less severe than those caused by fraud. Likewise, deficiencies that occur on an infrequent basis could be construed as less severe than those that occur consistently because recurring errors may be an indicator of a bigger issue, such as fraud or negligence.
• History of restatement. Deficiencies that lead to restatements are strong indicators of a material weakness. If a history of restatement exists, auditors could conclude that a restatement is possible if repeated deficiencies are identified, even though a restatement does not actually occur.

Box 5 also references the "prudent official" test. This test, which is performed by an independent investor, is used by internal auditors to analyze deficiencies from an investor's standpoint and to determine subjectively the deficiency's magnitude. Based on an analysis of this control, the overall impact of the deficient control is a control deficiency.

ITGC Deficiencies With Application-level Deficiencies
One of the significant issues with ITGC deficiencies is the potential for downstream impacts at the transaction process level; the Framework can be used to help organizations identify the impact of these deficiencies. Below is an example of how an ITGC deficiency could be pervasive to an organization and how it could impact the control's overall assessment.

First, assume that ITGC exceptions exist in application changes and that an analysis under Chart 1 finds that the exceptions are at least a control deficiency. Also assume that the exceptions relate to inadequate user acceptance testing and lack of approvals to move a newly implemented capital asset application into production. The major items that are processed by the application are depreciation expenses, gains and losses on asset disposals, plants, property and equipment balances, and accumulated depreciation.

During the course of the year, a financial statement audit discovered that the application miscalculated expense depreciations for more than six months and that the amount of the miscalculation is material to the financial statements. Based on these facts, an analysis under Chart 3 would look like this:

 

Evaluated ITGC Control: Application Change Control Chart 3: Box 1 Requirement: Are there complementary or redundant ITGCs that were tested and evaluated that achieve the same control objective? Answer: No Notes: The components of program change control (i.e., user acceptance testing and approvals for moves to production) are either inadequate or missing. No other complementary controls exist. If the answer to Box 1 is "Yes," go to Box 5. If the answer is "No," go to Box 2. Chart 3: Box 2 Requirement: Are there control deficiencies at the application level evaluated in Chart 2 that are related to or caused by the ITGC deficiency? Answer: Yes Notes: Based on the financial audit, an error was discovered related to the process in which the new application calculates depreciation expense. This error would have been caught if adequate user testing existed. This is based on a comparison between the new test application to the current legacy application in production. If the answer to Box 2 is "Yes," go to Box 3. If the answer is "No," go to Box 5. Chart 3: Box 3 Requirement: Are there control deficiencies at the application level related to or caused by the ITGC deficiency that are classified as only a deficiency? Notes: Refer to analysis on Chart 2.

Chart 3 for this example looks similar to the previous example with the exception that the answer to Box 2 is "Yes." In this instance, the internal auditor answered "Yes" to Box 2 because the depreciation expense calculation should have been detected with adequate user acceptance testing. In addition, Chart 3 requires the auditor to answer Box 3. To answer this box, an analysis of the depreciation expense calculation from a transaction process perspective is required. Charts 1 and 2 are used for this analysis and look like this:

 

Evaluated ITGC Control: Depreciation Expense Description: ABC application automatically calculates depreciation expense and records the applicable journal entry. Objective: To record the depreciation expense accurately. Risk: Depreciation expense is inaccurate. Chart 1: Box 1 Requirement: Examine and understand the cause and results of exceptions. Were the test's objectives met? Answer: No Notes: Based on substantive testing, an error was discovered in the processing and calculation of depreciation expense. Chart 1: Box 2 Requirement: Could additional testing support a conclusion that the deviation rate or observed exception is not representative of the total population? If any of the answers to Boxes 1 - 2 is "Yes," go to Box 3. If the answer is "No," a control deficiency exists; go to Chart 2. Chart 1: Box 3 Requirement: Extend testing and re-evaluate. Was the test objective met? If the answer to Box 3 is "Yes," it is not a control deficiency. If the answer is "No," a control deficiency exists; go to Chart 2. Conclusion for Chart 1: Control deficiency Chart 2: Box 1 Requirement: Is the potential magnitude inconsequential to annual and interim financial statements? Answer: No Notes: Depreciation expense was inaccurately calculated for more than one half of the year. The actual error and potential error are material to the financial statements. Chart 2: Box 2 Requirement: Are there complementary or redundant controls that were tested and evaluated that achieve the same control objective? Answer: No Notes: No other complementary controls detected the error. Chart 2: Box 3 Requirement: Are there compensating controls that were tested and evaluated that reduce the magnitude of a misstatement of both annual and interim financial statements to an inconsequential level? Answer: No Notes: No other compensating controls detected the error. If any of the answers to Boxes 1 - 3 is "Yes," go to Box 7. Otherwise, go to Box 4. Chart 2: Box 4 Requirement: Is the potential magnitude less than material to both annual and interim financial statements? Answer: No Notes: The actual error and potential error are material to the financial statements. Chart 2: Box 5 Requirement: Are there compensating controls that were tested and evaluated that reduce the magnitude of a misstatement of both annual and interim financial statements to less than material? Answer: No Chart 2: Box 6 Requirement: Does additional evaluation result in a judgment that   COMMENT ON THIS ARTICLE Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online, or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to edit/remove comments. Name: Email: Subject: Comment: To make something bold: <strong>Text to bold</strong> To make something italic: <em>Text to italicize</em> To make a hyperlink: <a href="URL">Text to link</a>          Home IIA Home Privacy Policy The Institute of Internal Auditors • 247 Maitland Avenue • Altamonte Springs, Florida 32701-4201 U.S.A. +1-407-937-1100 • Fax +1-407-937-1101 • www.internalauditoronline.org Copyright © 2013 The Institute of Internal Auditors Inc.