Internal Audit Upskilling for Critical AI Capabilities

For internal auditors, artificial intelligence (AI) upskilling is far more than mastering another technical skill. As many are already discovering, AI differs fundamentally from previous waves of technology, demanding new ways of thinking, learning, and working. To unlock the full potential of AI-powered tools, auditors must embrace both conventional training and less traditional approaches that build judgment, adaptability, and an understanding of how humans and intelligent systems work together.

Stefan Lundborg

One important consideration is the change in mindset that is necessary to make the most effective use of this tool. “It is leagues different from all other IT technologies we dealt with previously,” says Stefan Lundborg, former CAE at Örebro University in Örebro, Sweden, and a postdoctoral fellow in the research group at HEOS–Higher Education Organization Studies.

In the past, internal auditors set static rules for technology tools, and those tools moved systematically through them to generate predictable output, he notes. “But now we have AI models that almost appear human in their way of producing new data,” he says.

Use Cases for a New Paradigm

One of the many ways AI can be used is in quality assurance and enhancement, especially for small audit functions, according to Lundborg. Internal audit produces volumes of text, including audit plans, risk analyses, audit reports, and other written artifacts. AI is uniquely capable of analyzing this unstructured data and examining it for consistency, readability, gaps in reasoning, and other issues.

Previous tools could examine large, structured data sets, such as databases and financial records, but there were no tools to read through policy documents, reports, or similar, large text-based data, he notes. AI can thus significantly minimize the necessary time commitment for internal auditors.

AI can find anomalies and patterns in large unstructured data sets. It would have previously required extensive time or scope limitations to assess this data, Lundborg notes.

“AI tools can process large amounts of text in very little time, which also allows simple reprocessing of that same data in new directions as new questions arise during an audit engagement,” he says. AI can quickly review the same documents repeatedly to answer a range of questions about the material. “It is important to note that such use should complement rather than replace human analysis, to reduce risk of biases or hallucinations being introduced by the AI models,” he adds.

Internal Audit and Agentic AI

Given the rapid pace of technological developments, AI upskilling will necessarily require keeping tabs on new forms of AI and the use cases they offer. Internal auditors have gotten somewhat comfortable with various versions and uses of generative AI (GenAI), employing it for research or summarization, says John Romano, principal and internal audit and ERM service line leader with Baker Tilly. “But GenAI doesn’t really move the needle dramatically,” he says. “It’s more like a really good assistant.”

Agentic AI, which can work independently to reach multipart goals over a long period with limited human oversight, is a meaningful next step. Romano says he has seen some internal audit functions beginning to use it to review workpapers and perform a series of basic tasks. For example, based on instructions from the internal auditor, the program can perform tests, such as comparing invoices to purchase orders, checking that the related signatures and approval are appropriate, marking up the documentation, and providing a summary with its steps.

John Romano

His own organization uses an AI platform built for audit and advisory services at the front end of the audit, as well. Agentic AI requests information needed in the audit from the business owners. The AI request agent then reviews responses and alerts the business owner if more or different data is needed.

That can have a foundational impact, Romano says, because it takes away the friction in making repeated requests for information. AI agents also can collaborate on a project, he says. When internal auditors receive information from a business owner, one agent can review the requested information to confirm that it is complete and accurate, then pass it off to another that tests it and provides a summary.

Best Practices in Initial AI Upskilling

Nancy Hom

How can internal audit gain and maintain the necessary understanding of AI? Nancy Hom, vice president, Data, Analytics, and AI at MetLife, recommends a three-pronged approach:   

Learn AI.

• Leverage any enterprise-level AI training, along with tailored, role-based, audit-specific prompt engineering training to help establish a baseline knowledge of “how to talk to and delegate tasks to AI” for the audit department. “Setting this as required formal learning versus optional learning will accelerate the learning curve,” she says

Use AI.

• Once all auditors have gained the baseline AI skills, they are given the relevant AI tools to use across a menu of audit tasks.

Showcase AI.

• Intentionally feature case stories in various audit forums, spotlighting auditors using AI in their audit delivery, including the value, benefits, and challenges. “This natural exchange fosters experiential and social learning, and a positive learning culture across the audit department,” according to Hom.

The technology itself can help with maintaining current knowledge, Romano says. He recommends internal auditors ask AI to provide them with regular updates on new developments in AI overall and on its use in their industry.

A multifaceted approach can help internal auditors learn not only how AI can be used in their work, but also how it is being embedded into functions across the company by asking business owners about how they are using it. “Regardless of whether we’re using these tools in the internal audit function, our organizations are using them increasingly often,” Lundborg says.

Overall, internal auditors should have the chance to experiment with the new technology in real time, says Gina Adelphia, CAE at Jones Lang LaSalle, a global commercial real estate services and investment management company. Among other things, that will mean getting hands-on experience. Organizations will have to determine how best to let them do that in a safe environment, she adds.

Gina Adelphia

Perfecting the Craft

While traditional upskilling based on study and taking courses will be helpful, “it’s not going to be the catalyst that really will drive how you perfect your craft from this point forward,” Adelphia says.

“I think internal auditors sometimes mistakenly treat AI like it’s a search engine,” she says. While it is not necessary to be a data solutions engineer to work with it, internal auditors should become comfortable iterating with the technology, or keeping up a continuing dialogue until the auditor gets what he or she is seeking, she explains.

“It’s a behavior more than a skill, because the auditors have to get comfortable with almost treating that agent like a human,” Adelphia says. That means clearly describing what they want in a well-engineered prompt, then drawing the conversation out further to see how the AI reached its conclusions and challenging the technology to refine its output. “Your professional skepticism is really showcased on a day-to-day basis,” she says, as internal auditors assess the output they receive.

At the same time, internal auditors’ ability to make the most of AI depends in many ways on whether they are what Hom calls a “strong adopter.” These professionals not only understand the technology, but also know how to embed it into the audit workflow, she says.

That means understanding the best ways to use it throughout the workflow, such as to brainstorm and refine scoping, then asking it to suggest relevant controls and generate request lists for each control. After that, a strong adopter will ask AI to perform testing, reviewing documents based on criteria that the internal adopter defines.

AI then provides results and the final deliverable for the internal auditor to review. With agentic AI, this entire audit workflow could be orchestrated and performed with multiple agents, alongside the auditor, Hom says.

Strong adopters know how to delegate the right audit tasks to AI and improve audit speed and quality, according to Hom. That may seem like a challenge in some cases.

“A lot of the auditors built their careers taking great pride in designing a workpaper with beautiful tick marks,” Adelphia says. Internal auditors will have to accept passing that responsibility on to the technology so that they can focus on higher-value tasks.

Internal auditors also will have to develop a big-picture view of how AI is used across the organization. According to Romano, that involves understanding:

• How the company is using or plans to use AI and the level of its technical capabilities to handle the job.
• Where the points of risk, failure, and opportunity reside.
• The goals for AI uses.
• How AI might replace or augment current practices.
• Whether and how humans are being kept in the loop to monitor AI efforts.

Unfortunately, AI tools may not be well documented or well understood by those using them, Lundborg says, which can introduce biases and errors when the output is not critically assessed in relation to the limitations of the models. “Ultimately, the internal auditors will need sufficient knowledge and context not only of the models, but also of the processed data, to examine output that appears reasonable and recognize any errors, limitations, or other flaws,” he says.

Elevating the Profession

As businesses and other stakeholders rapidly expand their AI use, internal auditors face a certain urgency in upgrading their skills, according to Hom. “We have to match the organization’s pace of AI adoption, not just in auditing AI, but also in using it to align and enable enterprise strategy,” she says.

AI upskilling elevates the entire profession, Adelphia says. It takes on low-level tasks and enables internal auditors to use their critical thinking, business acumen, and professional skepticism in helping organizations harness an exciting and challenging new technology. “All those key skills become paramount,” she says.