control, and governance
The Three Cs of IT Change Management
Understanding an organization's change management culture, controls, and overall credibility will help internal auditors review IT change management policies and procedures more effectively.
Dwayne Melancon, CISA
Vice President of Corporate and Business Development
Properly established change management controls are the foundation of an effective IT control environment. Although considerable guidance exists on the proper ways to audit change management controls, many auditors often have trouble determining whether an organization's IT change management process is effective. This is the result of several challenges facing the internal audit profession. First, much of the audit guidance is in the form of checklists that don't properly identify which control activities are most important. Second, many audit guides predate new research that can help auditors focus on the right areas. Finally, many auditors are not equipped to challenge IT management effectively and do not fully understand their organization's change management issues.
To conduct more effective reviews of change management policies and procedures, auditors could use the "3-C" approach to change management — focusing on the organization's culture, controls, and credibility. The 3-C approach will not only help auditors fine tune their skills, it will also enhance their knowledge of change management fundamentals.
WHY AUDIT CHANGE MANAGEMENT?
Change management audits help protect the business from risk. However, in today's compliance-centric world, it is easy to lose sight of this simple, primary goal. For instance, many organizations implement policies and procedures simply to "pass the audit," without paying much attention to how policies are enforced. Worse still, some auditors base their reviews on static checklists that only allow them to mark the box providing the best answer, rather than conducting audits that accurately review the current IT landscape. The result is the approval of highly ineffective change-management processes and controls.
With this dynamic in mind, the information below will help internal auditors recommend effective and efficient change management best practices based on the 3-C approach. The goal of these practices is to encourage IT departments to operate in a way that protects the business from internal and external threats, while satisfying the organization's compliance requirements.
BEYOND CHECKLISTS: THE THREE Cs OF CHANGE MANAGEMENT
Auditors can determine the effectiveness of a change management system by testing three primary areas referred to as the three Cs of change management. The framework provided by the three Cs allows auditors to ask open-ended questions and enables them to apply their detective skills when conducting audits of change management processes and activities, even if they feel at a disadvantage due to a lack of technical knowledge. Below is a description of how to audit each primary area.
Understanding an organization's culture toward change management will help auditors obtain background information about the different change management processes and the company's overall attitude toward change. Key aspects of an organization's change management culture that auditors need to look out for include the company's:
Besides reviewing the elements above, auditors need to identify any potential red flags to determine if the organization's culture and attitudes toward change management might be in jeopardy. For example, some companies pay lip service to change management. When this happens, a change management process is in place, but employees are not held accountable for any violations, often because management doesn't know when violations occur. Another red flag is when the same types of outages occur repeatedly. This may signify that the organization is not learning from its past mistakes or that the organization is relying on individual knowledge to run the business, rather than building an organizational "play book" that allows them to establish repeatable processes and knowledge.
A final red flag is the ownership of multiple software products that perform similar functions. Purchasing different products to do the same work is often a sign that the organization is looking for a "silver bullet" solution to its change management problems. This often occurs because it is easier to justify buying new software than it is to drive a change initiative that alters how people and processes work.
Researchers at the IT Process Institute and Carnegie Mellon University's Software Engineering Institute have found that organizations achieve the highest leverage when implementing controls that center on change, access, and accountability. This allows organizations to focus on how work should be done within the company, who is allowed to do that work, and ensure that changes only happen within the organization's approved processes. By preventing personnel from circumventing change processes, high performers maintain "good IT hygiene" and encourage effective communication about change. This prevents the unexpected negative consequences associated with poorly managed changes, such as unexplained outages and high amounts of unplanned work.
Auditors can benefit from these studies by acquiring the necessary knowledge to help organizations establish controls that help to prevent, detect, and correct vulnerabilities and potential risk areas. For instance, auditors may review the effectiveness of IT controls aimed at preventing uncontrolled or unauthorized changes to the production environment. Below are examples of preventive, detective, and corrective controls:
After organizations implement the necessary preventive, detective, and corrective controls, auditors should ask questions that seek credible and demonstrable answers, such as:
Finally, auditors need to identify potential red flags that may affect established controls. A potential red flag might be the organization's inability to answer questions or provide credible evidence in a timely manner during the audit process. Unsatisfactory answers often hinge on "gut feelings," conjectures, time-consuming manual reviews of system event logs, e-mail-based approvals, and other inefficient methods.
Red flags also might include an inability to identify when changes occur outside the change management process. For instance, an organization might have preventive and corrective controls in place, but lack effective detective controls. Although these companies can report changes that go through the change management process, they are unaware of changes taking place outside this process until a problem occurs. Another red flag is spending a high percentage of resources performing unplanned work, such as fixing failed changes and responding to system outages.
Credibility is where it all comes together. In this context, credibility deals with the quality of the answers auditors receive when they test for effective control processes. This is the area where high-performing organizations shine: High performers understand their main goals are to protect the business from existing and unforeseen risks and operate in a manner that expedites the compliance process. In addition, organizations with high credibility have an established change management culture that sets clear organizational expectations on how corporate work will be conducted. They also have effective controls in place that identify how data will be collected and used. Finally, high performing companies have a system of checks and balances that helps people stay true to the change process and demonstrates there are clear consequences for noncompliance with internal regulations.
When testing for credibility, auditors need to ask questions that would be difficult for organizations to answer without established effective controls and a change management culture. During quality reviews, auditors may ask organizations questions concerning the following:
A common indicator of a credibility concern is the presence of a high volume of unauthorized changes, often relabeled as "emergency changes," which indicate that employees are not following established change authorization and review procedures and are finding loopholes to circumvent the approval process. High numbers of system availability issues (e.g., unexplained outages or security incidents) and increasing amounts of unplanned work also could alert auditors of the possibility of a faulty change management process. These symptoms are often the result of poorly coordinated change management practices and inadequate pre-production testing.
Another credibility concern indicator is the presence of unplanned work rates that exceed 20 percent to 25 percent. This typically indicates the organization is facing a culture or control problem. Late projects, cost overruns incurred from hiring external personnel to fix IT issues, and employee turnover also are indicators of systemic change management problems. Two final red flags include an IT department's inability to produce verifiable evidence that substantiates the presence of controls, as well as overly long response times when validating a control's presence or effectiveness. These indicate a lack of systematic tracking, documentation, and reconciliation of change authorizations, as well as a lack of post-incident scrutiny when service-interrupting incidents occur.
Moving beyond checklists by asking open-ended questions during an audit and probing for credible details is an effective way for internal auditors to avoid being "hoodwinked" during the review process. Applying deductive powers enables auditors to get to the audit's core purpose quickly. This includes verifying that managers are taking responsibility for their control environment and are behaving in a manner that protects the business from threats. To this end, auditors need to anchor their change management audits based on the company's culture, controls, and credibility, and coach their IT teams on how to implement control strategies that satisfy all three Cs.
During the past five years, there has been significant progress establishing a causal relationship between key IT controls and IT effectiveness. This research was started by the IT Process Institute (ITPI), a nonprofit organization that promotes the use of best-known IT practices. Other organizations have joined ITPI, including Carnegie Mellon's Software Engineering Institute, ISACA, and The Institute of Internal Auditors (IIA). Detailed information about ITPI's activities, including its IT Controls Benchmarking Survey and practices of high performing IT organizations, can be found on its Web site, www.itpi.org.
In addition, auditors and IT professionals should consult The IIA's Global Technology Audit Guide on change and patch management, Change and Patch Management Controls: Critical for Organizational Success. The guide can be downloaded free of charge from The IIA Web site, www.theiia.org/guidance/technology/gtag2/.
Dwayne Melancon, CISA, is vice president of corporate and business development for Tripwire Inc. Melancon has worked with ITPI conducting best practice research and with other corporations worldwide on IT service management improvement. Previously, he was vice president of professional services and customer support for Tripwire, as well as vice president of operations for DirectWeb.
COMMENT ON THIS ARTICLE
Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online, or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to edit/remove comments.