Measuring the Maturity of Risk Management

I am in the process of writing an article on assessing risk management and wanted to include an example of a maturity model. It would have to be one that is clear, and the vision embodied in the highest level would have to be something that I agree is both aspirational and achievable.

One source is the oft-referenced Carnegie Mellon University Capability Model. Another risk management maturity model resource is the Risk and Insurance Management Society (RIMS). The RIMS Maturity Model assesses defined attributes of the risk management program and places each at one of six maturity levels, from Non-existent to Leadership.

The maturity model I included (shown below) is derived from multiple sources, including the Chelan County Public Utility District, Washington. The risk management program as a whole is assessed based on five levels:

Level 1: Ad hoc. Undocumented; in a state of dynamic change; depends on individual heroics

Level 2: Preliminary. Risk defined in different ways and managed in silos. Process discipline is unlikely to be rigorous.

Level 3: Defined. A common risk assessment/response framework is in place. Organization-wide view of risk is provided to executive leadership. Action plans implemented in response to high priority risks.

Level 4: Integrated. Risk management activities coordinated across business areas. Common risk management tools and processes used where appropriate, with enterprise-wide risk monitoring, measurement and reporting. Alternative responses analyzed with scenario planning. Process metrics in place.

Level 5: Optimized. Risk discussion is embedded in strategic planning, capital allocation, and other processes and in daily decision-making. Early warning system to notify board and management to risks above established thresholds.

Questions:

  1. Do you like the model?
  2. Can you share a reference to a better model?
  3. What are your experiences using maturity models for risk management?
  4. Where does your program lie?

 

Posted on Mar 29, 2011 by Norman Marks

Share This Article:    

  1.  Congratulations on your working on an article t o assess risk management. I am sure it will be excellent and helpful to all!

    I do like the model above.

    Here is another model  from HM Treasury. I think it is a bit better because when I read  the terms used to describe the levels, I am easily able to relate this this to the latest risk management framework of ISO 31000, however the above model is also good. Perhaps both models can be combined to create a best in class model

    Awareness and Understanding-Level 1-Top management is aware of the need to manage uncertainty and risk and has made resources available to improve

    Implementation Planned and in progress- Senior managers take the lead to ensure that approaches for addressing risk are being developed and implemented

    Implementation in all key areas-Senior Managers act as role models to apply risk management consistently and thoroughly across the organization

    Embedded and Improving-Senior management is proactive in driving and maintaining the embedding and integration of risk management; in setting criteria and arrangements for risk management and in providing top down commitment to well managed risk taking to support and encourage innovation and the seizing of opportunities

    Excellent Capability established- Senior managers re enforce and sustain risk capability, organizational and business resilience and commitment to excellence. Leaders are regarded as exemplars

    Also see page 16 of the Hydro One Case- there is another example of a capability model

  1. Hi Norman,

    There are two you should consider, which we have discussed before on your blog:

    • UK's HM Treasury Risk Management Maturity Model, which is excellent in my opinion, and I have used it several times, and adopted it easily for commercial businesses http://www.hm-treasury.gov.uk/d/riskmanagement_assessment_framework220709.pdf
    • The IIA's Position Paper on Risk Based Internal Audit, which gives the Maturity Model Continuum: Risk Naive, Aware, Defined, Managed, Enabled, but very sparce definitions and no guidance on reaching that assessment.

    Regards

    Paul

  1. It sounds a good model. I always recommend a five level approach to try to nuance level of maturity achieved within the organisation. I know some consultant have their own model to assess this maturity level. We have used one once and it gives you a level or a sort of diagnostic or even measure of your integrated risk management with a Venn chart to show were weaknesses are identified. I keep thinking it is a good exercise to assess its risk management maturity level which implies the group global risk culture. It also gives you a starting point or a KPI you can then try to overperform and improve. No improvement without measurement I guess. Thanks for this document!

  1. Norman,

    I believe this is the perfect conversation for internal auditors to be having at this point in time. However, I think it is important to challenge the notion that risk management maturity lives outside 'management maturity' in general. I think internal audit can add more value by recognizing that there are elements of organizational maturity that can be more indicative of whether the organization will acheive its objectives or not...  Here are some types of organizational maturity I would be interested in measuring:

    Strategic/Business Objective Maturity - the formality by which strategic and business objective accountability has been disseminated and defined

    Management Control Maturity - the formality by which management creates the right objective monitoring mechanisms and controls

    Process Control Maturity - the optimization and alignment of the people, processes and technology required to efficiently and effectively reach business objectives  

    In my mind risk management is fully mature when it helps standardize the measurement of organizational maturity elements, and then reports threats in context of the organizational vulnerability given organizational maturity levels. 

  1. I am the most familiar with the Carnegie Mellon University Capability Model as it seems to be the most widely accepted standard of measurement.  However, I'm yet to find risk assessment guidance and / or modeling that really focuses on the acheivement one's objectives.  In the end, what's the point to risk if you're not measuring the achievement of your goals? 

    For example: if your objective is to stay at home and watch TV--than the risk of having a flat tire is not applicable.  Don't spend the time / money to fix your tire, if you really just want to stay home and watch TV! I know this is a very simple example, but the point is most Risk Assessment guidance and modeling does not emphasize the importance of understanding the impact to achieving the organization's objectives. 

    I propose the bigger question is: Is there a maturity model that emphasizes the monitoring of risks as it impacts the achievement of the Company's objectives?  Let's take this concpet one step further: Is there guidance that calls for the identification and monitoring of risk patterns as the patterns impacts the achievement of the objectives?  If so, that is where I'd place my vote! 

  1. Norman,

    I like your model. I tried to make one myself a view years ago. A that time I looked at the maturity model of Cobit. The result is very similar to what you present. A few differences:

    I also made the connection to selecting controls as aa result of risk analysis. At the higher levels there is a shift in preventing incidents in stead of reactive behaviour.

    Also at higher levels it becomes 'natural' to look at existing best practices and adapt usefull elements.

    I don't know if you see this as part of the framework, but in my model, at level 3 the responsibilities for risk management were explicitly attributed to functions (like risk managers).

     

     

     

  1. Norman: Very important area. Thanks for creating a forum to discuss.

    Connie: I agree with your perspective that risk management should have a focus on objectives. (implicit and explicit)   I have been working on a way to measure and report on the effectiveness/maturity of an organization's risk management framework now for 15 years.  The current version of what I propose as a tool to measure maturity/effectiveness is now posted on our website home page in the box titled "A better response from our risk experts in a bullet titled "Risk Fitness Quiz".  It can be found at  http://riskoversight.ca/.  It is what I refer to as a "objective-centric" framework. 

    I am also lobbying on a number of fronts to see if more debate and discussion can be generated to produce  some "authoritative" definitions of what constitutes "effective risk management processes" . The IIA December practice advisory flirts with it but never offers a solid definition of what constitutes "effective".  I am not a fan of binary opinion on risk managment processes.  

    I do agree that there is value in assessing and reporting on risk management maturity.  The RIMS risk maturity framework is probably the most advanced version of a maturity framework.

  1.  Tim:

    I am also working with a colleague to generate a document that can provide guidance on the assessment of  a risk management system. It will focus on the objectives of risk management, the principles of risk management, the risk management process and the risk management framework. So far the only document I have seen that gives this subject proper attention is the document from the UK- HM Treasury 2004. I am certain that we will see others as well take stabs at this this year.

     

  1. Norman, because certain types of risk may be managed well in some parts of the business and poorly in others, I would like to see a scorecard solution for identifying this scenario and then a roll up overall that is akin to the model you provided. In other words, an evaluation that shows what a company is doing well and where it has exposure. If in the process of the evaluation you could identify the value of return in the areas where risk is being managed well in contrast to the risk areas not being managed so well, you could conceivably perceive value from making improvements.

    Maybe I'm off track here and I'm thinking about a more sophisticated risk assessment of the risk management program, what do you think?

  1. Hi All,

    I would like to share with you my experience of the maturity model, the maturity model used in my work is also assessed on  five levels  as follows:

    1. Initiated .this level defined as controls are not appropriately designed to mitigate the risks, the characteristics of this level described as Inadequate or unsustainable design and  Incomplete ( Design stage )
    2. Defined. risks defined and controls are designed but not operating consistently, the characteristics of this level described as fundamental control design , inconsistent execution ,sustainability not demonstrated, and reactive( Design stage ) 
    3. Implemented. controls are designed and operated in a timely consistent and sustained way, the characteristics of this level described as control execution, policy compliant, accepted, understood , managed, consistent and timely ( Effective execution stage)
    4. Integrated. an efficient process with embedded controls that proactively addresses change, the characteristics of this level described as embraced, embedded,formally trained,leverages systems,fully documented,proactively manages business change ( Efficiency and sustainability stage )
    5. Optimized. a process that anticipates change and promotes continues improvement, this level characteristics described as anticipates change and continues improvement  (Best in class)

     

  1. It's difficult to find educated people on this topic, but you sound like you know what you're talking about! Thanks
  1. My brother suggested I would possibly like this web site. He was once entirely right. This put up actually made my day. You can not imagine just how a lot time I had spent for this info! Thank you!
  1. Norman,

     

    I like your model which is consistent with the one I have used before when working for one of the Big 4.

    However, one comment i would make is that the jump from level 3 - Defined to Level 4  - Integrated would be great and lengthy.

    Have you considered a timeline or level of expected maturity in relation to the existence of Risk Management in an organisation? 

    Russell  

  1.  Russell, I agree that moving up the maturity curve takes time. I believe the first issue for management is to recognize where they are and the continued opportunity for improvement.

    Some are content being only moderately mature, not recognizing the value if the higher levels. For them the time to mature is long.

    So, I don't have any "typical" times it takes to move up. Those that see the value and can find the resources will move faster than the "check the box" companies.

  1. To All; My observation, after working with risk  "maturity" models for over 20 years, is that word "maturity" is not really the issue.  Tp date. two alternatives that I think better address the issue are the level of risk management rigor/rigour (U.S. versus UK spelling) and using more everyday language, the level of "risk fitness".  Of the two, I think it is really the level of risk management rigor/rigour that best describes what really needs attention.  I don't believe "rigor" is the same as "maturity" using the common English definitions of the words.

     In the new approach we are promoting management, risk oversight committees and the board must make conscious decisions on the appropriate level of risk assessment rigor that think should be applied to key value creation and potentially value eroding objectives.  Internal audit can, and should, raise concerns in instances where they believe that management's decision on risk assessment rigor isn't appropriate. 

    Unfortunatey the majority of internal audit departments today, when they disagree with management's risk assessment rigor decision, elect to complete an assessment of control effectiveness instead.  This is akin to a parent not liking the way a child is approaching a task and, rather than teach them better ways to approach the task, just does it for the child.   This has been a key failing of traditional internal audit IMHO.

  1.  

    The HM Treasury RM assessment framework describes the two types of assessments it enables: Broad/Impressionistic versus Detailed. The terms hint at the trade-offs in the two approaches.
     
    I feel outputs of an effective maturity assessment need to give objective answers to "How good is the ERM program?", and "Which ERM areas to prioritize and improve on?" to whomever asks - Auditor, Board, Management, knowing that most people will never care for ERM as much as partitioners do.
     
    Toward those aims I highly recommend the "ERM Index (ERMi)" developed by the authors in this paper (http://www.ermsymposium.org/2013/pdf/erm-2013-paper-monda.pdf) .
     
    One key advantage of  this ERMi is that the questions and weighted scores that form the index were arrived at via Delphi method facilitated by the authors from a panel of ERM experts.
     
    The ERMi balances the trade-offs perfectly IMHO for the people that asks the questions. I do hope that it takes root and grows in our industry where consensus on what works is still difficult after so many years.

     

  1. Great conversation starter as always.

    I find a lot of the labels in maturity models to be benign, and reinforce a culture of mediocrity (going through the motions, looking good on paper, no real results).

    I've been toying with a scale of good, bad, ugly & great to focus the mind, with distinct differences between each.
     
    My definition of bad is contraversial.  For me, bad is when an organisation is compliant with all the relevant standards, and being able to pass an audit agaist ISO 31000 etc, but you have a strong feeling in your bones that you're not getting to the crux of matters.  In many models, my bad, would be labeled as mature.
     
    Also in the model, good is the opposite of great and all that usual good stuff.
     
     
     
     
  1.  hi all, I like to model and discussions and it fits as I need to come to some assessment of management awareness or maturity towards risk management. (it is a first baseline) and from this set an ambition level.

    Now I like to make some sort of questionair (max 20 questions) to give a first rough indication on where we stand in relation to IIA's levels  Risk Naive, Aware, Defined, Managed, Enabled.

    Does anyone already has such a questionaire that they can share?

  1. You might want to check out the P3M3 maturity model and self-assessment as it gives more than just a head-line assessment of maturity for project/programme/portfolio maturity.

    You might be more mature in some processes than others, and a single score doesn't reflect that or allow you to create a road map for improvement.

  1. And actually there is a maturity model in the appendices of "Management of Risk" (the OGC product in this area)

Leave a Reply