Revisiting Audit Reports and Ratings Based on a Report by E&Y

E&Y has published a thought-provoking piece in their 5 Insights for Executives series. This one is Internal Audit: a 3-D look at risk.

The opening scenario painted by E&Y is a scary one, given where I thought we have been moving as a profession. It describes a situation where internal audit is providing reports to the top executives and the Audit Committee that fail to provide basic information on whether the controls tested were “passing or failing”, the level of risk to the organization, whether control failures were previously known to management, and whether management is working to fix them.

Is this where many internal audit shops are? I certainly hope not!
 
Hopefully, we are all now moving (if not already there) to providing an opinion on whether the combination of controls in place is adequate to manage the business risks we are focusing on in the audit. In other words, the audit is the result of a top-down and risk-based approach, assessing whether risks are being managed within organizational tolerances. It’s about the management of risk, not just about controls.
 
The thought provoking part, for me, is the three-dimensional rating system. Of course, I would apply it to reporting on the management of risk and not the adequacy of controls per se.
 
My version of the suggested reporting system would include:
  • Management’s assessment of both the inherent risk (and maximum potential exposure) and residual risk levels. If I had to choose one to report, it would be residual and not inherent. My report would indicate whether I agreed with that assessment or not, and if not why not (which is where I would describe deficiencies in the design or operation of the combination of controls). My residual risk level would take into account an adjustment if any controls are not effective.
  • A trend line, showing whether risk management is improving or not.
  • Whether management will correct any deficiencies in a timely fashion. I have always followed a practice of acknowledging when management is already aware of an issue and working on it.
What do you think?
 
·         Is the scenario painted by E&Y realistic?
 
·         Have we moved to reporting on risk, rather than controls?
 
·         Should we report on the management of risk or the operation of specific controls?
 
·         What do you think of the 3-D risk rating system?
 
·         What does your ideal report include?

Posted on May 13, 2011 by Norman Marks

Share This Article:    

  Comments (11) un-categorized
  1. Comment by: Larry Brown   (5/14/11 01:12 PM)   

    Norman - Great post, and a thought-provoking piece, indeed!  I wouldn't be "alarmed" though, as you do a great job of living in both "theoryland" and "practiceland," and this slice of "alarm" comes mostly out of "theoryland."

    I think we'd all agree that there is no one right way to do things - i.e., can't call the doctor, can't call the ERM practitioner, can't call the GRC practitioner - but there are a range of acceptable practices in this area.

    I also think we need to stop whipsawing each other around the concepts of objectives, risks and controls as all need to be considered_together when making good business decisions (whatever the domain the decisions are being made in).

    Common language is great, all for it, let's keep trying to get there.

    Keep up the good work!

    Best,

    Larry

     

     

  1. Comment by: Lawrence Ellefson   (5/16/11 04:26 PM)   http://www.orcasaudit.com

    Norman....this is exactly why I developed ORCAS.

    I am retired now, but will state over and over, management needs to know what the level of risk is, what control procedures are mitigating them and how effective those procedures are. You can no longer get away with some general compliance statements. When I was an auditor (Arthur Andersen was the largest auditing firm on the planet, no one ever heard of Sarbanes or Oxley and Enron was just another energy company) things were different. Not now.

    Lawrence Ellefson

  1. Comment by: Michael Corcoran   (5/16/11 05:08 PM)   

    Norman, this is essentially a residual risk heat map (gross risk - risk response/treatment) = residual risk.  In the upper right quadrant IA is asked to help with risk response (consulting role). Upper left is where IA provides an (assurance role) testing that management actions are operating as intended. 

    I like it better because it does just not use the very traditional impact and likelihood exclusively. 

    Now imagine if management or IA had enough evidence to plot this for the enterprise and provide a point in time or annual risk and control environment opinion!

    Thanks

    Mike Corcoran

     

     

     

  1. Comment by: Jean-Louis Vergaert   (5/23/11 10:40 AM)    Norman, I think that E&Y is totally right in expressing how we, auditors, should think about a process, and all those elements must be included when we define our opinion: inherent risk, controls in place, residual risk, Management attitude and environment evolution. Whether we should formalize the documentation in a formal manner, and whether the Audit Committee or Top Management wants to micro-manage the Audit, is another matter. Making a formal report of all those elements takes time at the expense of auditing. I think that if the IA is made of experts in risks and controls, they should not have to explain and formalize all our assessments. However those elements must be included when we make an important audit finding. Now in practice: I was made aware of an external audit company who puts a red flag to our operation because "the journal entries" were not formally approved. After further investigation, the Finance Manager was actually the only one who entered the journal entries, and who had access to the General Ledger.... so much for documenting all that risk assessment but not investigating the practical circumstances.
  1. Comment by: Jacquetta Goy   (5/24/11 03:00 PM)   

    My organisation used to use a version of this type of heatmap (as recommended by a consultant). It gave very pecular results which senior management not surprisingly rejected. We've now moved to the ISO standard and are staring to get a better idea of the risk we need to manage. It is a pity that the Big Four audit organisations appear to have rejected the work of risk practitioners around the world in favour of inventing their own risk methodologies.

    The problem with this divergence is that risk and audit professionals rather than working together in a GRC type scenario to bring value to the organisation will instead produce conflicting results based on using different methodologies likely resulting in both disciplines being rejected by management teams and the risks not being managed.

    The approach outlined by Norman on the other hand is both complimentary and value adding. Risk management should be there facilitating management in making their risk assessments, whilst Internal Audit adds value in considering from an impartial view point whether the assessment is appropriate and the treatment plans are real or merely a paper exercise and also the overall progress and strength of the risk management approach. 

    I also think that E&Y should proof read it's articles better, assuming that they aren't really suggesting as good practice giving management credit for deficiencies they didn't know about as well as those that had been observed and plans put in place to address.

  1. Comment by: Mayurakshi Ray   (5/25/11 04:26 AM)   

    Norman, thanks for this article and yoru excellent insights.

    I'm afraid that most IA reports, modelled as process-control audits, don't report on risk management, atleast not formally in the reporting template. However, the consideration of inherent risk is almost always followed (in my experience) at the audit planning stages.

    I agree with you, that I would consider residual risk in my reporting, which would obviously give an opinion on the design and operating effectiveness of controls. In the reporting templates I've used, there is always a trend analysis / graphical representation on the process-control performance over time, a period-on-period measure of the improvement/degradation/status quo - which give the Audit Committee a clear picture of the business performance for each of the areas under audit. It would require a little rework to present the risk management effectiveness, but then, controls is just a part of risk management and thereby we are not in divergence anyways.

  1. Comment by: AMIT ROY   (5/25/11 06:36 AM)   

    Thanks Norman, for the article and your thoughts which are very well articulated.

    As very rightly expressed about the audit model  which should be based on process and system control and also express on risk management. Many of the corporates have started buliding the IA teams strength on the above domain. The expectation ,in particular of the 'Audit Committee' has also day by day becoming very high and therefore new oppurtunities and approach in audit function is getting elevated. In my organisation, we are following risk based audit  along with the process control approach. The reporting to audit committee and the auditee is also  risk based. The process control contributes tremendously towards better , efficient and stronger risk management. 

    IA fucntion is day by day getting more discussed in relation to risk management prospective which many of the corporates are adopting religiously. But I personally feel that though all such factors of control and risk as discussed are most important from reporting point of view to the audit commmittee and top management but I desire that the risk parameter identified through the risk heat map must be well expressed by the business group while considering any decision in business area. The awareness of risks management should get penetrated down the line in the business team. IA may give a  clear picture to the Audit Committee of the business performance for each of the function under audit but does the business performance improve on continuous reporting by IA.

  1. Comment by: John Schaefer   (5/26/11 05:34 AM)   http://transformationrmc.com

    I think that the focus needs to be on residual risks. This implicitly recognizes the inherent risk without getting into unproductive debate about the pure level of unabated risk. I also agree that reports must enable better decisions by understanding the cost and benefit of specific mitigation strategies (controls). Using earthquake risk as an example, it is fine to know that your operations are in a high risk earthquake zone (inherent risk) and also good to know that you have structural supports and a business continuity plan (the controls) but what is most interesting is how this affects ongoing business decisions such as whether or not you should expand operations at this site and what level of insurance you need. I think that unfortunately much of the valuable information gathered by auditors sits dormant after an audit report is filed and that few organizations have established processes to link audit work regarding risk assessment and control recommendations back to the business operations. This creates an opportunity for others in ERM or serving a similar risk role to gather and use data to help drive better business results.

  1. Comment by: arnold schanfield   (5/28/11 10:15 PM)   

     I like especially Jacquetta's comment that she believes that the Big 4 firms have rejected the established risk methodologies and invented their own and has stated what a pity it is. It is a pity. And we should all understand why  this has happened and remember it the next time any of the Big 4 firms  try to sell risk management services to our respective companies

  1. Comment by: Tom Harper   (3/30/13 09:02 AM)   http://www.fhlbc.com

     

    eport / control ratings are a communication mechanism and as such need to be designed to align with the culture and needs of the organization. We currently provide four ratings for each audit report: A compliance rating - is the area under review following formalized policies and procedures; An effectiveness rating - are the activities effective in managing the risks of the business; A design rating - this gets to the design of the controls - ie the controls could be well designed but poorly implemented. And lastly a governance & strategy rating - this reflects on managements active management of risks & controls and planning for future business changes.
    In our organization this allowed us to highlight the differences between complying with procedures and having well designed / implemented controls and to recognize a management that was more or less proactive in managing their risk environment.
    I think audit ratings in my experience are much more focused on by line management and often distract from the discussion of the real issues. To a limited extent the approach we have adopted has minimized some of that unproductive dialogue.
  1. Comment by: Todd Davies   (3/31/13 08:26 PM)   http://www.todddavies.com.au

     EY are onto something here, but I don't like their model, as it gives credit for plans which are sitting on a shelf and haven't been implemented. It's also needlessly complex in my opinion and risks baffling most readers. 

    I did push the idea of independent validation of management's risk assessment in the late 90s and spent a lot of time testing ways of doing that over the years. The conclusion I've come to is that this is a useful (and possibly essential) input into the audit conclusions, but these are the sub-detail to the conclusions. 

    For me, the debate about whether to report on risks or controls should have moved on. My preference is to provide / get assurance on objectives, and you need to assess both risks and controls to do that. 

Leave a Reply