control, and governance
control, and governance
August 2006
A Checkup for the Audit Shop
Quality assessment reviews help ensure audit operations and practices are running on all cylinders.
Neil Baker
Editor, Internal Auditing & Business Risk
Like many internal auditors, Andrew Levine had concerns when his audit shop prepared for a formal quality assessment (QA) last summer. He was just finishing his first year as assistant director of audit at the Port Authority of New York and New Jersey, but the department's senior management had been in place for years — 10 years in the case of its director. Had they kept up to date with best practice? Were they doing things right?
"A bad report would have begged a lot of questions," Levine says, "like 'What's wrong? Why are there so many recommendations?'"
|
Understanding the QA Requirement Given the 2002 start date, every internal audit department should have had a QA review by the January 2007 deadline. The exception is those shops that didn't exist in January 2002; they have five years from the date they were created. The standard allows for two approaches: a full external QA review, which can be carried out by a team composed of IIA volunteers or by a consultancy; and a self-assessment exercise followed by external and independent validation. Consultants say the self-assessment process is no soft option. Audit shops have to complete and document a rigorous process set out in The IIA's Quality Assessment Manual. This includes an assessment and gap analysis of how closely the audit activity follows IIA standards. A qualified external validator then reviews and tests the self-assessment work. The validator can either concur with the self-assessment opinion or write a separate opinion on the results of the process. The results are then shared with senior management and the audit committee. Audit shops that have not completed one of these routes by the January deadline will not be able to say that they operate in accordance with IIA standards. The change to the Standards is the most compelling reason why audit shops are establishing formal QA programs, but it is not the only one. Regulators, particularly in the financial services sector, are stressing the importance of the external assessment of internal audit work. The U.S. Sarbanes-Oxley Act of 2002 has put a spotlight on the extent to which management and external auditors can rely on the work of internal auditors, particularly around financial controls. And, more generally, as the profile of internal audit work increases inside organizations, the need to assure the quality of the audit shop makes obvious business.
|
In the coming months, other internal auditors will be hoping for a similar outcome. Under a professional standard introduced four years ago, audit shops are required to be quality assured by Jan. 1, 2007 (see "Understanding the QA Requirement" at right). As more of them get ready for an assessment — for some, their first in years; for others, their first ever — what can they do to make the process run smoothly, what common problems should they sort out now, ahead of time, and how can they ensure they get the most value out of the exercise?
KEEPING UP WITH BEST PRACTICES
Mary Ann Riesenberg explains clearly why she took her audit shop through a formal QA review. As director of audit and advisory services at the AARP, a nonprofit organization that represents the interests of people who are 50 and older, she wanted to comply with The IIA's International Standards for the Professional Practice of Internal Auditing (Standards) and find out whether the department was aligned with internal audit best practices. "We are constantly emphasizing to our audit clients the importance of keeping up with industry best practices," she says, "so it was only fitting that we follow our own advice and set a good example for others to follow."
The experience was positive. "The quality assessment team did an excellent job of keeping us up to date on the progress of its work and its preliminary findings," she says. "They recognized the positive aspects of our department and provided us with some helpful resources to refer to at the end of the project. Going through the review drove home the value of continuing to use these practices in our own audit projects."
To prepare for the external review, Riesenberg's team performed its own internal qa audit. This involved reviewing The IIA's QA manual to gain an understanding of what the external assessment would entail and organizing the information needed by the external review team. "This helped tremendously in getting ready for the external team, but next time, we plan to include more time between the two reviews, to allow time to address any improvement opportunities identified as a result of the self-review," she says.
Riesenberg also advises internal audit departments to make sure their policies and procedures, as well as any performance metrics they use, are well documented and up to date. In addition, she recommends giving people outside the department plenty of notice that the review is going to happen, so that they can make time to complete questionnaires and be available for interviews, if needed.
PREPARATION IS KEY
Alberto Ragazzini, internal auditing manager at Telecom Italia, Italy's largest phone company, also sees preparation as key to a successful assessment. Ragazzini, whose department was assessed for the first time in 2004, advises fellow auditors to plan carefully for the review and to take time to explain the regulatory environment in which the organization operates.
Ragazzini's audit shop — Telecom Italia Audit and Compliance Services — is a separate legal entity that is owned by two of the group companies that it audits; these companies are listed on the New York Stock Exchange and are affected by U.S. regulation.
"In our case, explaining the company's regulatory situation was really challenging," he says.
Levine, the port authority auditor, is another believer in preparation before the review. "Get your shop in order to the best of your ability before the review team's arrival," he says. "Do your own quality assessment review, review your workpapers, review your reports, make sure that your people are getting any training they need, and make sure you know what The IIA's Standards call for. You should be able to address those issues yourself beforehand, so by the time you're ready to have an actual review you should feel pretty sure that you're in compliance."
Although many internal auditors stress the importance of preparation, not everyone puts in the effort. "First impressions do make a difference," says Basil Woller, leader of the firm-wide QA program at consultants Protiviti Inc. Some audit shops get their client material organized in a systematic fashion, have their stakeholder interviews organized, and get all the logistics taken care of, while others treat it "like a fire drill," he says.
Woller advises internal auditors to arrange the review material as they would for an audit. "It makes the work of the reviewer more efficient and effective, and it ought to give you some assurance that you have been thorough, systematic, and comprehensive in terms of your preparation," he says. "And as you go through the preparation process, you will invariably come up with what I call 'aha' moments, when you see things you need to get in place before the review team comes."
Woller has seen many organizations take that a step further and commission an independent readiness review, particularly if they are concerned that they do not comply with the standards. They will either bring in the same consultants that will carry out the QA review, or appoint an independent one, "for a quick check or assessment to identify any significant gaps," he says.
Annette Mumford, vice president, chief audit officer, and compliance officer at Homestreet Bank, recommends providing audit staff with some training in quality assessment before the review starts. "That helps them to understand the quality assessment objectives, scope, and process," she says. Before her own group's review, she also made sure the full department attended the first and last meeting with the review team. "That was important in keeping everyone involved and invested in the process and outcome," she explains.
As a small audit shop with only six employees, the team at HomeStreet, one of the largest private banks in the U.S. Pacific Northwest, had to decide whether to go for a full external quality review or to take the self-assessment route. "I chose the full review, as it would go beyond compliance with The IIA's Standards and include an assessment of our shop against best practices," Mumford says. She also felt a full review was "clearly a more independent assessment."
The review team was constructive and helpful. They reinforced the benefits of some changes Mumford was planning already, such as automated workpapers, and they recommended improved benchmarks and metrics for internal audit activity.
Mumford says the exercise was very worthwhile. "We had a knowledgeable review team; all three had experience as chief audit executives, and two had prior quality assessment experience," she explains. "Having a strong team allowed us to have confidence in their work and added credibility to their feedback."
AN UNEASY FEELING
Despite the comfort level many auditors have with their external QA teams, the experience of being on the other end of an audit may involve a degree of trepidation. "Having another group look at your department is sort of odd because you're no longer in the power seat," Levine says. "But the people who came were very qualified and we had a good rapport with them, so those fears went by the wayside pretty quickly."
The uneasiness Levine describes is a common feeling among internal auditors, according to Celia Elmey, a consultant who performs qa reviews of internal audit departments in the United Kingdom for accountants Ernst & Young. But breaking through it can be a valuable experience, she says. Nobody really likes to be audited, so turning the tables and making an auditor into an audit client for a while can be a good leveller. "They are reminded how uncomfortable it can feel to be under the spotlight, and that the auditor's approach and personal skills can determine whether the experience is a positive or negative one," Elmey says.
Getting a perspective on what you are doing is always valuable, especially when it comes from one of your peers, she says. Chief internal auditors are often starved of that input, because they don't have any direct peers in their company, she explains. When they have a quality review, "they get an opportunity to bounce ideas around and compare what they are doing with other companies."
An independent reviewer is also in a good position to provide an additional perspective — that of stakeholders. When an audit shop has a full quality review, "they learn what stakeholders really think of them," Elmey says. That objectivity can be lacking in a self-assessment approach. "When someone else comes in and asks the stakeholders what they think without anyone from auditing there, you often can draw out different and more honest views." She says this is often the most valuable part of the entire exercise.
COMMON PROBLEMS
Elmey says most reviews result in mainly positive feedback for the department. "Broadly speaking, people working in internal auditing are good at what they do and understand their jobs," she says. "I don't think I've ever come across an audit team that I thought was absolutely doing the wrong thing."
When Elmey sees weaknesses, they tend to be in nontechnical areas. The quality of audit reporting is one of her main areas of concern. Regardless of how good the audit work is, "people only judge you by the reports they receive," she explains. "Lots of audit committee members tell me they can't see the wood for the trees."
She also sees problems with the wrong mix of skills and experience in a department. Sometimes the organization wants to use internal auditing as a training ground for management, which can be a good idea, but can cause problems. Taken too far, it can result in high turnover or a team staffed with too many juniors. At the other end of the scale, an audit shop full of long-serving professionals can lack new ideas. The trick is landing somewhere in the middle, which is difficult, she says.
CONSTRUCTIVE CRITICISM
Inevitably, the review team will spot areas that can be improved. The key, according to those who've undergone an assessment, is to be receptive to that feedback and any recommendations offered.
Levine advises auditors to be honest and open. "Look at it as an opportunity," he says, "and take any recommended improvements to heart. Don't look at it as adversarial. Welcome them in and work with them." Don't be like those audit clients who gripe about internal audit recommendations, he says.
Being open about his department's weaknesses helped Ragazzini get more value out of the QA experience. "Nobody's perfect, and the review team will not expect to find a perfect situation, so don't pretend to be perfect," he says. "The review is an occasion to solve your problems. In our case we 'disclosed' in the preliminary meeting what our main weakness was and that we were already working on it, and we got valuable suggestions from the review team."
A LEARNING OPPORTUNITY
The overriding message for audit shops thinking about their QA review seems to be: prepare, be open, and be positive — treat the exercise as an opportunity to learn. Elmey and Woller, both experienced reviewers, say they rarely come up with major surprises. Usually, if something needs to be worked on, the internal auditors will know about it already. Where auditors benefit is having an independent view on how they are doing, and some suggestions about moving forward. "What you get is an infusion of fresh ideas," Woller says. "That does not mean you have to adopt them, but it's a great opportunity to get better in your organization." The review is about quality assessment, but the purpose is to improve, not criticize.
To comment on this article, e-mail the author at neil.baker@theiia.org.
Internal Auditor is pleased to provide you an opportunity to share your thoughts about the articles posted on this site. Some comments may be reprinted elsewhere, online, or offline. We encourage lively, open discussion and only ask that you refrain from personal comments and remarks that are off topic. Internal Auditor reserves the right to edit/remove comments.
To make something bold:
<strong>Text to bold</strong>
To make something italic:
<em>Text to italicize</em>
To make a hyperlink:
<a href="URL">Text to link</a>
