AI Tools Customized for Internal Auditors

Organizations are adopting AI at a remarkable pace, and their internal audit functions are following suit. According to research from the Internal Audit Foundation and AuditBoard (now Optro), 83% of senior internal audit leaders expect their function to increase its use of AI over the next year.

Yet plans to use the technology do not guarantee that internal auditors understand it, are comfortable applying it, or know where to find reliable guidance on using it. Because internal auditors provide assurance over risk, these knowledge gaps will remain a critical concern.

Practical guidance is available, however, with more coming. It covers AI from both a risk management perspective and an internal audit implementation perspective

Looking to IIA Resources

The IIA has created a suite of tools designed to support internal auditors and stakeholders in their alignment with The IIA’s Artificial Intelligence Auditing Framework. The IIA AI Framework Support Tools are concise, targeted, easy-to-use documents that members can put to work right away, says Andrew Guasp, IIA director of Standards and Professional Guidance — IT & AI.

Andrew Guasp

Nick Lasenko

“They respond to members’ immediate need for practical information and tools related to AI,” he says. “Because AI is a pervasive technology that can affect the entire organization, The IIA has committed to helping internal auditors not only understand how to audit it but how to use it within internal audit activities.”

With their open-ended questions and accessibility, The IIA tools “give auditors a real-life flavor of the governance issues involved,” says Nick Lasenko, a cybersecurity advisor through Vaco by Highspring supporting Cox Enterprises, and a member of The IIA Standards & Guidance: AI Knowledge Group. He notes that while many of the available AI-related standards and control lists are either very high-level or very granular, the IIA’s AI tools provide practical, actionable information in a concise format. “They’re a bridge between a high-level framework and a detailed engagement plan,” he says.

In addition, the tools contain timely guidance that internal auditors need to enhance their roles. “When we master generative AI, we can get it to do the heavy lifting of the mundane and repetitive parts of our roles,” says Pete McLeod, associate director, Internal Audit, at The University of Queensland, Australia, and also a member of The IIA Standards & Guidance: AI Knowledge Group. “That provides an opportunity to achieve more. We can refocus and engage with our stakeholders, learn and keep abreast of the business, and move into the trusted adviser space.”

Brainstorming Done for You

Four of the five tools are presented in a questionnaire format, containing relevant questions in a variety of key areas, as well as practices and notes that help internal auditors understand each question and the information necessary to answer it. These tools include:

Pete McLeod

• ERM: Management AI Risk Assessment Questionnaire. Recommendations for ERM professionals on how to determine whether management is appropriately identifying, evaluating, managing, and monitoring AI risk.
• Board AI Governance Questionnaire. Questions for internal auditors evaluating the board’s oversight of AI governance.
• Internal Audit: Management AI Risk and Control Questionnaire. Questions designed to help internal auditors gain a structured understanding of how management identifies, assesses, manages, monitors, and reports on AI risks across the organization.
• Audit Committee: Internal Audit AI Coverage Questionnaire. Sample questions for audit committee members to use in assessing whether the audit plan has sufficient AI-related coverage.

The overall goal is to provide examples in all the areas in which internal audit is involved, Guasp says. “The tools do much of the work that internal auditors do in planning meetings, risk assessment meetings, and other front-end efforts” he says.

The questions in the ERM, board AI governance, and management AI risk tools cover topics including governance and culture; strategy and objective-setting; performance; review and revision; and information, communication, and reporting. The Audit Committee tool, meanwhile, addresses audit plan coverage and risk alignment, scope, objectives and engagement design, and all aspects of the internal audit role necessary for the audit committee to understand.

Collectively, these tools can give internal auditors greater confidence they are asking the right questions, Guasp says. The examples are not intended to be all-inclusive in every situation, but they can help internal auditors get started in the right direction. “We did some of the brainstorming for you,” he says.

For instance, the first sample question in the Board AI Governance Questionnaire tool delves into how the board has defined and communicated its risk appetite for AI use. A related example delineates what to look for in the board’s risk appetite statement, including use cases, prohibited or restricted AI applications, and requirements for human oversight. Further questions on this topic examine the board’s understanding of the organization’s current AI oversight, the existence of an AI governance framework or policies, and executive accountability for AI governance, among other topics.

These tools also help internal auditors think about issues in the context of AI use, such as risk, accountability at enterprise and individual levels, organizational readiness and the completeness of AI inventories and audit coverage, Lasenko says. McLeod says he has even seen internal audit teams using the Management AI Risk and Control Questionnaire and ERM: Management AI Risk Assessment Questionnaire to help management develop its AI governance framework in an advisory role.

Using an Audit Lens

Another IIA tool, AI Prompting for Internal Auditors, is designed to help internal audit functions leverage AI themselves. There are prompts for planning, fieldwork, testing, audit analysis, and reporting, along with tips for creating strong prompts. According to McLeod, who has a PhD in AI, internal auditors are using prompts to:  

• Create risk control matrices.
• Develop executive summaries from report findings.
• Evaluate whether there is sufficient evidence for design and operational effectiveness, evaluation, and conclusions.
• Examine the possibility of creating AI agents for reuse.

Ross Visscher

Teams should always consider the risk of hallucinations when working with AI, McLeod notes, and expand their skills to include data analytics so they can validate AI output when using it. The IIA tools “reaffirm parts of what we are doing by ensuring proper data governance, documenting prompts, and following good processes,” he says.

Ross Visscher, audit manager (IT & analytics) for the city of Calgary, Alberta, Canada, has received good feedback on the tools from his team. They found the prompts a useful, audit-specific starting point for their own efforts.

For example, in a section on chain-of-thought prompts, the example prompt is:

“Think step-by-step about the key risks in an organization’s vendor onboarding process, considering both financial and operational risks. After completing your analysis, summarize the top five risks as audit objectives.”

The tool provides explanations of what the prompts are meant to achieve. For example, in this case it notes that this will prompt AI to “work through relevant factors before generating the final list, producing a more thorough and structured result than a direct request would,” says Visscher. “This isn’t just generic prompting guidance. The prompts help you consider the use of prompting in accordance with your professional responsibilities. They put an audit lens on AI, which makes the tools really relevant.”

Working Toward Mastery

Lasenko notes it is important to realize that even the newest best practices may not cover every consideration. “There are new risks emerging, and just because one hasn’t been codified yet doesn’t mean it isn’t important,” he says.

Separately, when using AI to analyze internal information, he advises internal auditors to remember that AI may not reliably identify which source document is authoritative. Knowing this, internal auditors should provide context on which sources are current, approved and applicable to the task.

When preparing inputs, internal auditors should keep in mind that unnecessary detail can lower the output quality.  Large inputs consume more of the model’s context window and can leave less room for reasoning and the response. For a focused question, Lasenko advises: “Instead of giving it a 300-page PDF, all the AI may need is one of the pages.”

Taking the First Step

The AI landscape can easily feel overwhelming, but that feeling does not have to last. Resources such as the various AI tools provided by The IIA will quickly help internal auditors better position themselves and their organizations to take advantage of technology’s vast potential.

“GenAI is a pervasive technology that has the ability to transform the way we work,” McLeod says He compares AI mastery to the progression of carpenters, who begin as novices and work their way up to making beautiful furniture as an artisan — all while using the same chisel, mallet, and miter saw. To begin the journey to AI mastery, internal audit must take the first step.