Change Control Audits — A Must for Critical System Functionality

March 2007

Change Control Audits - A Must for Critical System Functionality

Reviews of an organization's change control environment will help internal auditors detect critical system failures and risks before they occur.

Francis Kaitano, CISSP, MCSD, MCAD.Net
Assistant Advisor, Technology and Security Risk Services
Ernst & Young

Pressure to make rapid changes to IT systems without a formal review process often results in a critical system failure due to unforeseen technical problems or the use of inadequate risk analysis and testing. Although they may be narrow in scope, internal audits of an organization's change control policies and procedures provide management with assessments that identify whether the controls present adequately mitigate existing and future risks. As a result, internal auditors should analyze and review the organization's change control environment to determine if changes to critical systems follow established company guidelines and are in line with regulatory requirements.

REVIEWING THE CHANGE CONTROL ENVIRONMENT

IT audits usually consist of independent evaluations of companywide policies, procedures, standards, measures, and practices that safeguard electronic information from loss, damage, unintended disclosure, and denial-of-service attacks. An important component of many IT audits is the review of an organization's change control environment. Simply stated, change control is the process used to request, review, specify, plan, approve, and implement changes to a system. When it's properly implemented, change control assures that unplanned changes don't happen and that planned changes are well managed.

As part of the change control process, IT professionals need to understand that all changes to an application or any business-critical system must go through a formal, standardized process. The internal auditor's role is then to determine whether the necessary controls are in place according to the organization's policies and internal and external regulatory mandates, as well as identify whether established controls are implemented properly and are adhered to by company staff. Once the audit takes place, the auditor should report any shortcomings to management for action.

Documentation Is Key
During audits of an organization's change control process, auditors need to determine whether the company's change management system is working as intended. An effective change management system must document all changes, whether these changes consist of fixes, enhancements, or major revisions. As a result, it is essential that any change to the application or system is initiated first by a request that is documented, reviewed, and approved by the appropriate staff. In particular, the documentation of the change management process needs to specify who is responsible for performing the following five roles:

Who can request changes?
Who can approve changes?
Who can develop changes?
Who can test the changes for compliance with approved specifications?
Who can move the changes into production?

When reviewing answers to these questions, the auditor needs to identify whether the same person is responsible for each these (i.e., Is there an appropriate segregation of duties to ensure the process is not compromised?).

Besides the approval process, the auditor needs to determine if a change impact review was completed. Any issues found during the change review activity must be addressed through a back-out plan for the specific change that describes how a failed change can be restored to its previous state. Because back-out plans help IT departments restore systems to their last known working condition, they should be a part of the organization's overall IT policies and procedures. Performance and security impact reviews also should be conduced to evaluate whether a change is adversely affecting the control risk.

Generally, all changes made to the organization's IT infrastructure, including emergency changes, should be tracked by an automated change management application or other documented procedure. Because emergency changes represent the greatest risk to an organization, IT departments need to make sure these changes are not implemented with lower scrutiny. Therefore, an emergency change process must be in place to handle situations in which the normal approval process impedes the business requirements placed on the application. The role of the auditor is to review the emergency change to determine whether it was approved, implemented, and complied with by company staff.

Furthermore, if the organization uses an automated change management product to track changes, the auditor should have access to the system and determine whether changes are being tracked. In addition, auditors should request all testing documentation on each change and verify how the testing process was conducted. To identify how the testing process took place, auditors should ask the following questions:

Can changes be made without using the automated change management application? Auditors should keep in mind that some changes do not require the use of the change management tool. These kinds of changes should be allowed as long as they don't affect the change management system's functionality.
Are there excessive permissions on the system? If the answer to this question is "yes," the danger of unauthorized changes is likely to increase.
Is there a detective control (e.g., a change scanning tool) that can identify all changes? If the answer is "no," then there is little assurance that all changes are truly known. On the other hand, if the answer is "yes," auditors need to determine whether the detected changes are reconciled with an authorized change.

The important point to remember during audits of an organization's change control methodology is that detection and reconciliation procedures should exist and be documented, ideally using an automated change management tool.

Testing Changes
As part of the change management process, companies may use different testing procedures, such as user acceptance tests (UATs), integration tests, and regression tests. UATs are an important control aspect because they determine whether an application meets agreed-upon requirements. UATs are performed by users of the application during the final stages of a project prior to final companywide distribution. Integration tests, on the other hand, take place prior to the final stages of the project when different parts or modules of an application are combined to determine if they function together correctly. Finally, regression testing identifies whether the application's effect on other business processes is being monitored effectively. More specifically, this kind of testing determines if:

Any system malfunctions were fixed.
Previously working functions have not failed as a result of the reparations.
Newly added features have not created problems with previous versions of the software.

Regardless of the type of testing performed, all tests should be commensurate with the risks identified. Auditors, therefore, should pay close attention to the types of tests conducted, the results of each test, and whether or not test recommendations were implemented. For instance, an organization may have one change process with multiple models to manage several change processes as a single unit.These models are used mostly by engineers and programmers to manage ongoing development of digital documents such as application source codes and other critical information that may be worked on by a team of people. One model may require regression testing, some may require integration testing, or all may require user acceptance testing.

Furthermore, auditors should test designed controls to help ensure that programmers cannot make changes to the source code's production version. Auditors also need to check whether separate test environments exist so that development, testing, quality assurance assessments, staging environments, and production activities are segregated by their individual functions. In particular, auditors need to review whether:

The source or object code is synchronized with the production code.
A software release process is documented and in place.
The source code is secured and versioned appropriately.
The code is managed effectively.

Finally, auditors need to verify all user sign-off tests. For example, the auditor needs to identify whether the documentation is stored in a secure area, including test plans, user sign-offs, and step-by-step instructions or "walkthroughs."

THE CHANGE CONTROL PROCESS

Due to the diversity of risks that may arise with a poorly implemented change control process, internal auditors should determine whether the organization's change control environment includes, but is not limited to, the following components:

A quality system methodology to ensure business requirements are met.
A written change control procedure.
A change request form.
One or more change review teams.
An online validation function.
A production installation plan with an uninstallation contingency plan.
A quality assurance group or function.
Appropriate segregation of duties.
An appropriate permissions model.
Retention of applicable documentation.

When reviewing the minimum components above, auditors need to check whether the organization is keeping all change requests and test results for an appropriate time length. This timeframe can be determined by industry best practices or can be based on regulatory requirements. During the review process, the auditor should verify that a proper risk analysis takes place and the following change control procedures are implemented:

Change requests are submitted using the established protocol.
All changes are evaluated based on established criteria.
Validation assessments are conducted periodically or whenever a major system change is implemented to ensure change requests are submitted and evaluated appropriately.
Implementation plans are created and carried out, including an uninstallation contingency plan.
Changes are tested before being rolled out.
Changes are approved prior to completion.
Changes are executed and qualified (i.e., changes have been thoroughly tested and a proper risk analysis is done to deem the changes suitable for implementation).
Changes are approved prior to closing the original request.
Special treatments of emergency changes are documented.
Special treatments of routine or standard changes are documented.

After change control procedures are identified and assessed, the auditor's role is to conduct an extensive review of the change process' available documentation and verify that the process was done in a way that mitigates business risks. First, the auditor needs to evaluate the design of the process and determine if it is effective in mitigating risks while supporting the needs of the business. If the change process is implemented in a way that does not mitigate existing and future risks, the company may end up with poorly implemented controls. Second, the auditor needs to assess compliance to the process.

Following the review process, the auditor should provide recommendations that improve any weak areas in the change management process. The recommendations listed below represent some of the main suggestions auditors can present to business owners during reviews of the company's change control process:

A proper risk analysis should be done for each requested change that takes into consideration any impacts to the organization.
After the risk analysis takes place, a business impact analysis should be conducted. Impacts to the organization should be one of the primary risks considered to determine which change model to use.
The review and quality assurance team should ensure that internal controls are implemented in a way that effectively mitigates business risks.
Changes implemented should have a demonstrable business benefit and, where appropriate, a quantifiable return on investment.
Business owners should establish policies and procedures restricting programmer access to the production software and data to an emergency basis only.
Business owners should implement a consistent approach to software library management, including a restrictive segregation of duties approach for programmer and analyst access to the application and production data and systems.
The IT department needs to establish a process for documenting how it will control access and track all activities performed, including those taking place during an emergency.
Change control procedures implemented should ensure that the organization adheres to all regulatory compliance mandates and standards.

Finally, auditors can recommend that the organization uses industry or published best practices to create company-specific processes that include proper controls and risk mitigation. These best practices should assist the IT department in establishing and maintaining standard business procedures and internal controls for changes to the organization's software, hardware, and network system. Because these best practices may not be all inclusive, management should review all business procedures to identify risks and establish the controls necessary to mitigate those risks.

LOOKING FORWARD

In today's world of rapid IT development, change management is a top concern for many organizations and internal auditors are becoming a cornerstone in every IT department. New legislation and competitive global economics are forcing organizations to have the controls necessary to mitigate internal and external risks. As a result, it is essential that IT professionals and internal auditors work together to understand the concepts of risk and control more effectively and determine whether business objectives are met in an effective and appropriate manner. A well-executed change control process lowers the risks associated with business-critical systems, thus making sure the success of the organization's business strategy and goals.

For additional information, auditors should read The IIA's Global Technology Audit Guide on change management, Change and Patch Management Controls: Critical for Organizational Success, which can be downloaded free of charge from The IIA Web site. Auditors also can refer to the chapter on change management in the IT Infrastructure Library's Service Support Book, developed by the UK's Office of Government Commerce.

Francis Kaitano, CISSP, MCSD, MCAD.Net, is an assistant advisor on Ernst & Young's Technology & Security Risk Services in Zimbabwe. Prior to working with Ernst & Young, Kaitano was a systems and solutions developer at CIMAS, a private medical aid society in Zimbabwe. Kaitano also has worked independently as an industry analyst and researcher, covering IT compliance, risk analysis, applications development, incident response, and computer forensics issues.

Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online, or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to edit/remove comments.

To make something bold:
<strong>Text to bold</strong>

To make something italic:
<em>Text to italicize</em>

To make a hyperlink:
<a href="URL">Text to link</a>