Date posted: 19/11/2026

How to check AI's work

Verifying what AI gives you, in proportion to the risk.

In brief

  • AI usually sounds just as confident when it is wrong as when it is right.
  • AI gets things wrong in two common ways: it makes things up, or it gives an answer that is right in general but wrong for your situation.
  • How hard you check comes down to two things: how well you know the topic, and how much is at stake.
  • The work, and the accountability, stay yours. “The AI told me” is never an answer to a client, your manager or an auditor.
  • Human review remains essential. AI can also help check work, but its findings need assessment too.

By Inbal Rodnay

When we stop checking

What happens when your team leans more and more on AI, and somewhere along the way the checking goes quiet? AI's reliability goes up. Your alert level goes down. Work moves faster than ever. Then one day, something wrong goes out with your name on it.

That is over-reliance, and it is a significant risk with AI. As AI becomes more capable and familiar, and better at following your instructions, it becomes easier to wave the work through.

This is already happening. In KPMG and the University of Melbourne’s 2025 study, more than half of employees who use AI at work in Australia (57%) and New Zealand (51%) said they had relied on AI output without checking its accuracy. The pull behind it has a name: automation bias, an excessive trust in automated systems that leads to less critical thinking and oversight.

Here is how it can play out. CA ANZ and ACCA’s 2026 research describes a hypothetical finance team whose AI tool automates most of its month-end close. Over time, the team stops spot-checking the tool’s “match confirmed” results. A small, recurring exchange-rate difference slips through and, over several quarters, adds up to a $2 million misstatement by the time the auditors find it.

Whatever the tool, it is still your name on the work, not the AI’s. However good AI gets, it doesn’t earn blind trust. This article looks at the two common ways AI gets it wrong, a grid for deciding how hard to check, five ways to check more deeply, and the emerging practice of adding an AI review as well as a human one.

Confident, and still sometimes wrong

AI always sounds like it knows. Perfect grammar, calm and certain, beautifully articulate. None of that tells you whether it is right.

The first way AI gets it wrong is the most familiar: it makes things up. The first way AI gets it wrong is the most familiar: it makes things up. It may invent facts, figures, citations or sources, or present uncertain information as though it were established. This is often called a hallucination.

A related problem is outdated information. An answer can look polished and plausible but rely on an old threshold, rule or assumption. In accounting work, one incorrect fact or assumption can undermine the conclusion built on it.

The second way is even easier to miss: right in general, wrong for your situation. The AI was missing context, or quietly assumed it. Picture a client asking AI directly: "What is the most tax-effective way to take $180,000 out of my company this year?" They get a clean comparison of salary against franked dividends. Correct as far as it goes.

But they never mentioned the family trust in the group, the unused concessional cap, or the Division 7A loan that changes the picture.

They never said which country, so it assumed one. For a New Zealand company, the missing pieces might be imputation credits or an overdrawn shareholder current account.

The answer to the question they asked was right. But the question wasn’t enough.

If you were asking, and tax planning is your area, you’d mention the missing details because you know they matter. Or you’d read the answer and quickly notice what was left out. Same question, same AI, very different risk.

The difference is the expertise of the person asking.

So the first thing to work out, every time, is whether you are the expert here.

How much should you check?

Not everything needs the same level of checking, and checking everything to the same depth isn’t practical. The depth you need comes down to two things: how well you know the topic and how much is at stake. Treat as high stakes anything that leaves your organisation, feeds reported figures or a tax position, or will inform a significant decision. The verification grid below shows how they combine.

Verification grid graphic

Image credit: Image supplied by Inbal Rodney.
  1. You know it, low stakes. AI tidies your file note or drafts an internal recap of a meeting you ran. Read it for accuracy and completeness, adjust the tone so it sounds like you, then file it.
  2. You know it, high stakes. AI drafts a client advice letter or a board paper on a matter you know well. You can spot the errors, but your name is going on it. Read every line and every number, and rework it until you own it. Check the facts and assumptions against the source material, and recalculate figures where needed. Don’t paste AI’s work straight out, even when you know the topic. Reworking it is how you verify it.
  3. Not your area, low stakes. You ask AI to explain FBT on electric vehicles so you are oriented before a client or management conversation. Nothing is going out yet. Ask where it got the information, read the relevant source and use a second AI tool your organisation has approved to look for gaps.
  4. Not your area, high stakes. A cross-border transaction outside your specialty, a new accounting standard you have not applied, or a fresh ruling you have not worked with. This needs the full treatment: all five checks below, including a human expert, before anything goes out.

Five ways to check more deeply

When it is your area, your expertise helps you review the work, but the depth of checking still depends on the stakes. Review it as you would a junior colleague’s work, but be extra wary of confident specifics: figures, dates, thresholds and citations. Is it factually right? Does anything feel overconfident? What is missing? Read it aloud, make it sound like you, and remove generic phrasing and anything you would not say yourself.

When the stakes are high or it is not your area, treat AI as a research starting point, not the answer. Slow down and go deeper:

  1. Ask for the sources. Open them yourself. Check that they exist, are current and support the answer. A citation is a starting point for checking, not proof that the answer is correct.
  2. Surface the assumptions. Ask what it assumed and what else it would need to know. This often changes the answer entirely.
  3. Change the angle. Ask again with the AI playing the client, manager or reviewer who will receive the answer: what would confuse them, what would they challenge? If the substance shifts a lot, slow down.
  4. Cross-check with another AI. Ask another approved AI tool the same question. Look for facts that disagree and caveats one gives that the other misses. Different tools can also make the same mistake, so agreement does not establish that the answer is correct.
  5. Sense-check with a human expert. For high-stakes work outside your expertise, this is the final test: before it goes out, have someone with real expertise in the area check it. That could be a colleague, a specialist, an external adviser or a trusted peer. See: What your AI policy has to cover.

How it is changing now

Until now, the rule has run one way: a human reviews AI-assisted work before it is relied on or shared.

Another approach is to add an AI review, using an approved tool, before work goes out, where it can help identify errors or omissions. Not instead of the human. As well as the human. Clients, managers and boards may also run your work through AI themselves and come back with a report suggesting something was missed. An AI review can help you pick up those questions first. Like any AI output, its findings need checking before you act on them.

And AI tools are starting to do it themselves. Microsoft’s Researcher agent in Copilot, for example, can have one AI model draft a report and a second model review it before you see it.

None of this changes who is accountable. If you cannot explain how every number was worked out and what the assumptions were, the work is not ready, however many AIs touched it. You can never say “the AI told me”. Not to a client, not to your manager, not to an auditor. Keep a short record of what you checked and how, especially for client-facing or audit-relevant work.

Your next move

Before your next piece of AI work goes out, put it on the grid. Two questions. Do I really know this topic? How much is riding on it? Let that guide the depth of your checks. If you find gaps, conflicting evidence or unresolved questions, investigate further.

And one move for your team or peers. When AI gives you a wrong or half-right answer, share it. It makes the risk real for everyone, and everyone's guard goes back up. That keeps everyone alert, not just the careful few.

Sources

ACCA and CA ANZ, Enabling finance insight: Bridging skills and data gaps for AI-enabled finance, July 2026

KPMG and the University of Melbourne, Trust, attitudes and use of artificial intelligence: A global study 2025 (Australia snapshot), 2025

KPMG and the University of Melbourne, Trust, attitudes and use of artificial intelligence: A global study 2025 (New Zealand snapshot), 2025

Microsoft, Use model choice in the Researcher agent, 2026

Disclaimer
This guide provides general guidance only and is current as at October 2026. Tool names, features and capabilities were correct at that date but may change. The publication therefore focuses on general principles rather than specific feature lists.

This guide is not intended to be comprehensive and is not a substitute for professional, legal or other advice tailored to your circumstances. Use it alongside your organisation’s artificial intelligence policies and your applicable legal, regulatory, professional and ethical obligations. Check current vendor terms, product information and your own obligations before acting on or relying on the information.

While Chartered Accountants Australia and New Zealand (CA ANZ) seeks to ensure that the information is accurate and current, it does not guarantee its accuracy, completeness, currency or suitability for any particular purpose. To the fullest extent permitted by law, CA ANZ disclaims all warranties and liability for any direct, indirect, special or consequential loss or damage arising out of or in connection with any inaccurate statement, opinion, advice or information contained in this publication, any omission from it, or the use of or reliance on this [guide/document] or any material it contains.

© 2026 Chartered Accountants Australia and New Zealand ABN 50 084 642 571

Search related topics