M.A.I. Consulting
AnalysisAdvanced29 September 20266 min read

Five questions every board should ask about AI this quarter

Trustees do not need a technical briefing. They need five questions asked every quarter, with evidence. Here is the Quarterly Five, with the answers that should worry you.

Most boards handle AI in one of two ways. Some receive an annual presentation from a vendor or an enthusiastic executive, nod, and move on. Others discuss it as a regulatory risk in the abstract and never look at what their own organisation is doing on a Tuesday.

Neither is oversight. Oversight is asking a small number of the same questions, every quarter, and reading the answers with the same seriousness as the financial report. Trustees do not need to understand how a language model works. They need to understand what their organisation is doing with one, and whether anyone could show them.

Why a fixed set of questions beats a deep dive

Board time is scarce, and technical briefings age quickly. A standing set of questions has three advantages. It builds a trend line, because the same question answered four quarters running shows direction. It protects against charm, because a confident presenter cannot steer a fixed agenda. And it teaches management what the board will ask, which changes what management prepares.

We call our version the Quarterly Five. Each question is deliberately answerable with evidence rather than reassurance, and each has a tell: a typical answer that should raise concern.

The Quarterly Five

None of these requires a technical background. All of them require management to have done some work before the meeting, which is the point.

What good answers look like

A good answer to Question 1 is a one-page inventory with a named owner, a last-reviewed date and a note of how it was compiled (survey, log review, conversation). A good answer to Question 3 is specific. One published study of 758 consultants, run with Harvard Business School researchers, found that people using AI completed tasks about 25% faster and produced work of higher quality on tasks inside the tool's capability, but were 19 percentage points less likely to reach the correct answer on a task deliberately placed outside it. The lesson for a board is not that AI is unreliable. It is that the benefit and the risk are both real, and that only measurement on your own work tells you which side of the line a given task is on.

A good answer to Question 4 includes something uncomfortable: a draft that was wrong and was caught, a document that nearly contained data it should not, a decision that was reversed. Organisations that can produce these are not the ones with more problems. They are the ones that can see the problems they have.

A board that only hears about AI when it works has been given a sales report, not an oversight report.

Setting up the routine

A quarterly routine can be introduced without any new committee.

  1. Add a standing agenda item. Twenty minutes, placed with risk or finance, not with strategy or innovation.
  2. Agree a one-page template. The five questions as headings, with management's answers beneath and a trend indicator (better, same, worse) against last quarter.
  3. Name the presenting executive. One person answers for the whole page, so the board is not left to assemble it from three departments.
  4. Ask for one example in full. Once a quarter, pick a single AI-assisted piece of work and walk the board through it from request to sign-off.

Two cautions. First, resist adding questions until the five have run for a year; a long list turns oversight into a compliance exercise. Second, do not delegate the question set to management. If management writes the questions, they will be answerable with reassurance.

The minimum a trustee needs to know

There is a modest body of knowledge worth having, and it fits on a page. Trustees should be able to say what a language model can and cannot do in plain terms, understand that outputs can be fluent and wrong at the same time, know that anything typed into a tool may leave the organisation's control unless the contract says otherwise, and recognise that a policy is not evidence of practice. That is enough to ask the Quarterly Five well.

Questions for your next leadership meeting

The bottom line

Board oversight of AI is not a technical briefing but a quarterly routine: where it is used, what data went in, what it cost and returned, what went wrong, and who signs. The Quarterly Five gives trustees a fixed, evidence-based set of questions and a way to spot the answers that do not hold up.

Prepared by M.A.I. Consulting.

Sources

Series · AI Consulting: leadership briefings · part 3 of 11
Leadership & Perspective

Where does AI sit in how an organisation is run?

If this is the question on your desk, a thirty-minute call tells you whether the service fits, or that you do not need us yet.