December 2011

External Assessments as Tactical Tools

A public sector internal audit function uses its external assessment to deliver strategic value to the organization.

Christopher B. MacDonald, CIA
Audit Principal
Citizenship and Immigration Canada

In 2011, we set out to validate the results of our internal self-assessment, assess conformance to internal audit standards, and assess progress against the objectives of our audit policy via an external assessment. The process enabled us to take a strategic look at our function and create a forward-looking document to plan the next generation of internal audit within our organization. We used the external assessment as a strategic opportunity to maximize the benefit to the audit function and to the organization.

WHO WE ARE

The Internal Audit and Accountability Branch at the Department of Citizenship and Immigration Canada was established in 2002 with the hiring of a chief audit executive (CAE) and one staff member. It now comprises 12 employees and is headquartered in Ottawa, Ontario. The department’s mission is to build a stronger Canada by developing and implementing policies, programs, and services that facilitate the arrival of people to Canada; safeguard refugees and people in need of protection; enhance the values of Canadian citizenship; and foster increased intercultural understanding and an integrated society. The department has an annual budget of approximately US $1.6 billion and 4,759 employees stationed in Canada and throughout the world. The department has 46 points of service among five domestic regions and 86 points of service in 73 countries, and the internal audit risk-based planning process has been effective in focusing on key initiatives and in providing assurance on all high-risk areas of the department’s operations.

In 2006, the Canadian government established and implemented a new audit policy, which is based in large part on the International Professional Practices Framework. This marked a profound change in the overall delivery of internal audit and assurance services. The two most critical changes were that CAEs had to report directly to their organization’s deputy head and that audit committees had to be composed of a majority of external members.

FROM VISION TO ACTION

As part of the long-term plan for the internal audit function, and in response to the new audit policy, an external, independent team performed a preliminary quality practice inspection in 2006 to assess internal audit’s level of conformance to professional audit standards. The inspection identified strengths and areas for improvement to ensure that internal audit’s work would continue to add value to the department while at the same time meeting or exceeding internal audit standards. The audit committee was presented with the assessment results and an action plan to address opportunities for improvement was developed.

From 2007 to 2010 the chief audit executive (CAE) led the continuation of the plan to fully implement the audit policy while the function focused on delivering high-quality, value-added reports. This included addressing opportunities from the preliminary practice inspection and building a quality assurance improvement program within the branch. During this time, the audit branch grew to its current size and includes the CAE, an audit director, and one quality assurance (QA) manager. The QA manager supported the growth of the internal audit function by standardizing practices and processes and including them in a comprehensive internal audit manual, conducting regular quality reviews of internal audit workpapers, and undertaking continuous improvement initiatives to support auditors in the consistent application of the audit manual.

Since 2007, the branch has produced 32 internal audit reports that have gone through rigorous management oversight — from on-site supervision to quality reviews carried out by the branch QA manager. Based on the results of these reviews, the quality program has focused on developing and implementing new tools for auditors and enhancing key governance pieces such as the departmental internal audit charter, departmental audit committee charter, audit manual, and audit reporting templates. In addition to these items, annual performance self-assessments of the function are performed internally to ensure that progress is being made against key risk-based audit plan commitments.

EXTERNAL ASSESSMENT FINDINGS

Our approved Risk-based Audit Plan 2010–2013 includes several steps that had to be taken before conducting an independent external assessment. The first was conducting the internal self-assessment, which was done in mid-2010. The second was hiring an independent, external resource to conduct the external validation exercise.

As stated in Practice Advisory 1312-1: External Assessments, internal audit functions can choose to undergo either a full external assessment or have an independent, qualified reviewer conduct an independent validation of an internal self-assessment. We decided to conduct a rigorous self-assessment of our function and have an external validation of the results performed. The IIA or one of many other qualified firms can conduct external validations. According to PA 1312-1, the leader of the assessment should have a combination of internal audit competence, audit management experience, QA training, and previous experience working on external quality assessments. Our external validation exercise was completed last February by a Big Four firm and presented to the audit committee in April.

The external validation report concluded that the branch generally conformed to The IIA’s Standards and Code of Ethics. It also identified areas that could be further strengthened. The branch was deemed to partially conform to Standard 2200: Engagement Planning and Standard 2400: Communicating Results. Observations indicated that the branch could further strengthen audit engagement planning in the areas of explaining scope exclusions, better documenting audit engagement risk assessments in workpapers, and improving the quality of communications in audit reports.

LESSONS LEARNED

The independent, external validation of our internal self-assessment highlighted several lessons that internal audit will carry forward in future assessments. By understanding what worked well and what did not, internal auditors can maximize the value and cost-benefits of the lessons.

Experience and Communication

The value that the independent assessment team brought was experience — both in the internal audit field and in conducting independent reviews of other internal audit functions. In addition, establishing open, regular communication among the CAE, the internal audit project manager, and the external assessment team proved to be of great importance. It encouraged openness to the assessment and facilitated the timely exchange of information. Based on the experience of the assessment team and the open communications that were established, we were able to gain insight into best practices of similar audit functions and develop an action plan to take the internal audit function toward the next generation of audits within the organization.

Planning and Coordination

Like the audit itself, the time spent planning the assessment exercise was critical. If the self-assessment was not done correctly it could have impacted the time that it took to perform the external validation exercise and decrease the value that it brought to the internal audit function. As such, managing a project of this magnitude required patience, good internal documentation practices, and good communication skills. From an internal audit perspective, coordinating the provision of information and having documentation to support the self-assessment was key. Internal audit found that having solid QA practices in place and a well-documented internal self-assessment contributed to the success of the validation exercise.

External Validation vs. Self-assessment

In some cases, the results of the branch’s internal self-assessment did not match those of the independent reviewer. This was the case for Standard 2200: Engagement Planning and Standard 2400: Communicating Results, where internal audit self-rated as “generally conformed” while the external assessor rated these as “partially conformed.” The external validation report included a table that summarized all standards, the ratings, and where the ratings were different. Where the results differed, the independent reviewer offered detailed explanations as to why this was the case. This feedback helped us consider how best to approach future self-assessments, and it served as a road map for responding efficiently to the recommendations for improvements.

Staff Survey Insight

Another useful component of the validation exercise was an independent survey of the branch’s internal audit staff. This provided auditors with a chance to answer questions on their abilities and experiences, and it has led to improvements in internal training programs and how the branch manages and conducts audits. The members of the audit committee found this part of the assessment worthwhile, as it provided them with the audit staff’s view of the internal audit function. The results of this survey were informative and have led to changes in how internal audit training will be planned and managed, as well as how individual auditor performance objectives will be linked to the overall audit plan and action plan to address the assessment results. A follow-up survey of internal audit staff conducted by an external party will be conducted more regularly.

Senior Management Interviews

Perhaps the most significant contribution was having an independent reviewer meet with and interview a cross-section of the most senior managers within the organization, officials from our external auditor (the Office of the Auditor General of Canada), and members of the audit committee. While the branch’s function conducts post-audit surveys at the completion of each audit, these interviews provided an opportunity for an independent reviewer to get a more complete picture from various stakeholders on their perspective of the internal audit function. The results were invaluable. They confirmed our self-assessment and post-audit survey results and provided a good starting point for developing the next generation of internal audit and assurance work in our organization.

STRATEGIC VALUE

Since the external assessment, the CAE has taken several steps to address the opportunities raised in the report on the state of the function. These steps include updating the internal audit manual to further enhance processes and procedures, having more frequent all-staff meetings that focus on sharing information about ongoing audits and process improvements, and integrating the results of the external assessment and the associated action plan into the performance objectives of individual audit staff members. Finally, a formal, two-day training program was developed to address the areas of partial conformance, particularly as they relate to audit engagement planning. 

There is strategic value in having an external assessment conducted of any internal audit function. External assessments provide CAEs the opportunity to obtain an independent assessment of their function. As such, the results should be used tactically and as a starting point for reflecting on how the internal audit function can be enhanced to achieve maximum impact for the organization. For the branch’s internal audit function, it has led to the initial conceptualization of a model for the next generation of audits at the Department of Citizenship and Immigration Canada. With the strong, continuing support of the head of the department, senior management, and audit committee members, the branch has a solid basis from which it can launch more strategic discussions.