Reaching Compliance Through Foundational IT Controls

Effective use of key IT controls - or foundational IT controls - is necessary to provide cost-effective risk management for the organization.

Dwayne Melancon, CISA
Vice President of Corporate and Business Development, Tripwire Inc.

While many companies have been lost in the maze of IT frameworks, best practices, and suggested controls, a recent IT Process Institute (ITPI) study revealed there is a small number of key controls that, when implemented properly, will fuel an organization's compliance, security, and IT service quality simultaneously. ITPI's IT Controls Performance Study examined 96 IT groups from various industries to determine whether the Pareto principle, often referred to as "the 80-20 rule," applies to IT controls. The Pareto principle states that for many phenomena, 80 percent of the consequences stem from 20 percent of the causes.

After three years of research, ITPI discovered that a small percent of IT controls, also known as foundational controls, provide a disproportionately high amount of coverage. In fact, a majority of the value gained from IT controls was derived by implementing a controls-based framework, such as ISACA's Control Objectives for Information and related Technology (CobiT). As part of their work, internal auditors review the effectiveness of IT systems in reducing risk and protecting customer and business-critical information. Learning what these foundational controls are can help internal auditors provide recommendations that will enable companies to manage risk more effectively, while meeting compliance mandates.

FOUNDATIONAL CONTROLS DEFINED

As part of the study, ITPI identified a group of high-performing IT organizations - companies that have best-in-class performance on operational measures, security activities, and compliance indicators. ITPI's initial hypothesis was that these organizations were using a common subset of IT controls. More specifically, ITPI hypothesized these high performers discovered that certain IT controls simultaneously impacted IT compliance, improved operational efficiency and effectiveness, and reduced potential business risk.

To find which controls mattered most, ITPI, in partnership with researchers from Carnegie Mellon University, Florida State University, and Tripwire Inc., identified the preferred places where high-performing organizations first implement their IT controls. The result was a list of 63 CobiT controls spanning the six leading areas within the UK's Office of Government Commerce IT Infrastructure Library - access, change, resolution, configuration, release, and service-level management. To analyze the service impact of these controls, the survey included 25 performance indicators spanning audit, operations, and security performance measures. These indicators included IT user satisfaction, unplanned work, security effectiveness, and audit compliance disruption levels.

Researchers compared the control activities performed by the study participants with the performance measures they reported and searched for a relationship between the two. The result: The Pareto principle does apply to IT controls. According to the study, each of the six control categories could be reduced to either three or four foundational controls, which have the same impact on performance measures as the full set of controls. A total of 21 foundational controls were identified in the six control categories combined.

Through this analysis, the researchers were able to determine which organizations had the best risk management posture. Figure 1 describes this process. On the vector diagrams below, each wedge represents one of the foundational controls. The size of the colored shading within each wedge represents the percent of the cluster members that responded "yes" to that control. A marked contrast can be seen between the high and low performers: Almost all organizations represented in the high-performing cluster had all of the foundational controls, while almost all of the survey respondents in the low-performing group had no foundational controls, except for access and resolution.

Figure 1. Diagram of low, medium, and high control clusters

What the researchers found was that medium and low performers were using a variety of controls in different areas and were not focusing their energies in the right places. Therefore, they were not using the foundational controls necessary for maximum effectiveness. This lack of focus hindered the benefits received from each of the controls.

The top two foundational controls most present in high performers and least present in medium and low performers were:

A systematic means to monitor systems for unauthorized change (ITIL area: change).
Defined, enforced consequences for instances of intentional unauthorized changes (ITIL area: change).

Because these controls were correlated strongly to high organizational performance, they were considered key levers in the high performers' control set. In fact, these two controls are "discriminant" controls - that is, when they are absent, the organization is never a high performer. The next four foundational controls most present in high performers and least present in the other classes of performers were:

A formal process for IT configuration management (ITIL area: configuration).
An automated process for configuration management (ITIL area: configuration).
A process to track change success rates (ITIL area: change).
A process that provides relevant personnel with correct and accurate information on current IT infrastructure configurations (ITIL area: configuration).

It is important to note that these top six foundational controls relate to change and configuration management.

The study found that these six controls are necessary for organizations to manage business risks effectively, because they enable the organization to avert risky changes ahead of time and identify the source of any outages or service impairments caused by change. A configuration management process that tracks and records change success rates and the root cause of service-impacting events provides strong value in the change and incident management processes. In other words, high performers use processes that enable them to rule out change as a cause early in the diagnostic process. Maintaining an accurate change history allows IT to understand what changes were made to which configurations and remediate quickly any changes that result in a negative incident. The ITPI's Visible Ops Handbook (2005) - a book that describes the best practices of 11 high-performing IT organizations - refers to this approach as fostering a "culture of causality."

Furthermore, this approach enables organizations to use past results to drive future activities. For example, if a particular type of change has exhibited a low success rate based on historical change success data, the organization can take additional actions - such as better pre-implementation testing and more rigorous change review activities - to improve change success rates in the future. The presence of historical data enables organizations to measure the effectiveness of these process and policy changes.

FACILITATING COMPLIANCE

While conducting the study, ITPI found that high-performers outperformed everyone else by a significant amount, often demonstrating five to 10 times better performance than the other groups. The following examples illustrate how high performers prevented the introduction of unnecessary risk to the organization:

While high performers authorized and performed five times more IT changes than medium performers, and 14 times more changes than low performers, they had half the change failure rate of medium performers and one-third the change failure rate of low performers. Furthermore, the percentage of unplanned work in top performers was 12 percent lower than in medium performers and 37 percent lower than in low performers.
When high performers had a security breach, the breaches were less likely to result in loss events, such as financial, reputation, or customer losses. Loss events in high performers were 29 percent less likely than in medium performers and 84 percent less likely than in low performers.
High performers detected security breaches better than medium and low performers through the use of automated controls. Compared to high performers, medium performers were 60 percent less likely to detect a breach through the use of automated controls, while low performers were 79 percent less likely to do so. In other words, high performers had the right controls in place to detect security breaches. On the other hand, low performers would discover the breach from an external source, such as a customer or a newspaper headline.
High performers were better able to detect security breaches quickly. High performers detected breaches in minutes, compared to hours for medium performers and days for low performers.

Why is the ability to detect variance and document changes important in an IT environment? According to the study, high-performing organizations that detected variance and documented changes quickly managed risk more effectively because they had an informed view of their production environment. In addition, these organizations were able to provide proof that management audited the actual practices of organizations and enforced accountability for process and policy adherence. On the other hand, low performers were unable to provide credible proof that they had the means necessary to detect unauthorized changes in their IT environments, which could have resulted in the organization's exposure to unknown risks. Because an unknown risk cannot be managed, the business will likely suffer as a result. From a risk management perspective, the study found that high performers had a lower compliance cost, fewer audit findings, and fewer security incidents than medium and low performers.

MOVING FROM GOOD TO GREAT

Without a methodology or culture for controlling change, unauthorized changes will destroy the effectiveness of an organization's IT department. By not controlling change, the organization sets in motion a downward spiral that produces unplanned work, mediocre service quality, a high mean time to restore service(s), ineffective security, and a poor compliance posture. The following metrics are useful IT audit, operations, and security measures to determine how well an IT department is performing:

Amount of time devoted to unplanned work. Unplanned work rates that exceed 20 percent to 25 percent typically indicate culture or control problems. According to the Visible Ops Handbook, high performers should spend less than 5 percent of their time on unplanned work.
Volume of emergency changes. Are unauthorized changes called "emergency changes" as a way to get out of disciplining employees for violating prescribed processes or as a means to circumvent formal change management procedures? High performers tend to classify less than 5 percent of their changes as emergency changes. Any emergency change rates of more than 15 percent are often a warning sign to auditors that change controls are being circumvented.
Number and causes of failed changes. The study found that high performers were able to sustain change success rates of more than 95 percent, and some as high as 99 percent, as defined by the changes that were implemented successfully without causing an outage or unplanned work episode.

Warning signs of systemic change management problems in medium and low performers included:

A high frequency of unexplained outages, security incidents, and other system availability issues.
A history of late projects and cost overruns, incurred by contracting resources to compensate for unplanned and emergency work.
High employee turnover and employee morale issues.

To transition from a low or medium performer to a high performer, the IT Controls Performance Survey reveals that the first step is to create a culture of change control and a culture of causality. To use foundational IT controls, the organization must establish the proper tone at the top - that is, all change must follow the change management policy from the highest levels of the organization to the lowest levels. To be successful at this foundational step, a policy of zero tolerance for unauthorized change must be established and promoted clearly and consistently by executive management.

Executives must establish concrete consequences for violating internal processes and policies, while actively enforcing them. Many high performers in the study instituted a "warn once, discipline on a repeat offense" practice. To ingrain the importance of this policy early on, many of them involved top management in the warning process. As a result, a solid, written change management policy is fundamental because it establishes the necessary groundwork needed for a culture of change management.

As a key step, the change management policy should establish a governing body - sometimes called a change advisory board - to review all changes and evaluate each for risk prior to approval. It is then the board's role to reinforce the written policy, which should include mandatory testing of all changes and a rollback plan for each change in case it causes an unexpected result. The auditor should check to determine whether the policy allows exceptions to emergency changes - although at times there may be a streamlined approval process, there should still be a process.

Effective change management also requires post-incident reviews to instill a culture of learning so that the organization reduces the likelihood of repeated mistakes. Change owners should document their findings and integrate what they've learned into operational practices. In addition to a defined change management policy and specific consequences for circumventing this policy, there needs to be visibility into all change activities, not just authorized changes. High performers use automated controls that avoid the need for constant human vigilance, reducing the risk of human error and staff expense to manage the process. Specifically, auditors should determine if the implemented technology ensures good coverage of all the foundational controls. Different types of technologies include:

Preventive. This type of technology uses a change management or authorization system (e.g., the use of the IT service or help desk) that guides the change process, tracks status of changes, and creates an audit trail of authorizations.
Detective. This type of technology uses an automated, independent, detective control or random change audits to monitor the production environment for changes, compare changes with authorizations, and detect "out-of-band" changes - undocumented changes that circumvent the change review and authorization process or violate organizational policy. Out-of-band changes also include "extra changes" performed under the cover of an authorized work order.
Corrective. This type of technology implements processes (e.g., provisioning systems or backup and restoration programs) to restore a system to a full-service capacity or return it to a known, supported state when unauthorized changes are detected.

FOUNDATIONAL CONTROLS ARE ESSENTIAL

For any organization struggling to find the holy grail of IT controls as a way to satisfy IT risk and operational efficiency and effectiveness, the ITPI data is clear: A majority of the value driven from IT controls can be derived by implementing a small fraction of CobiT or any other controls-based framework. These general IT controls are imperative for effective IT operations, because they span the bounds of multiple compliance domains and are imperative to reducing risk, while at the same time increasing IT efficiency and effectiveness. In addition, the results of the ITPI's research prove that controlling change and creating a culture of change management and causality can transform low- and medium-performing organizations into a high performer, delivering more value to the business with less risk.

Dwayne Melancon, CISA, is vice president of corporate and business development for Tripwire Inc. Melancon has worked with ITPI conducting best practice research and with other corporations worldwide on IT service management improvement. Previously, he was vice president of professional services and customer support for Tripwire, as well as vice president of operations for DirectWeb.