OneNym.idDiscuss deployment
English

THE BOARDROOM FIREWALL / Chapter 10

The AI Question

The same AI tool can make an analyst far more productive in the morning and help a fraudster imitate the finance director's voice in the afternoon. This chapter helps leaders see where AI creates risk and opportunity, and which decisions only leadership can make.

This page summarises the chapter. The full text is in the book.

Three ways AI touches security

The book describes AI as a gifted new tenant in the building: useful in almost every office, but naive about what is confidential, sometimes confidently wrong, and available for hire by outsiders who mean harm. Leaders do not need to understand how large language models work. They need to keep three kinds of AI risk separate. The first is AI as a tool staff use, where the concern is what information leaves the company and whether outputs can be trusted. The second is AI as a weapon in criminals' hands, making old frauds more convincing. The third is AI the company builds or buys into its products and processes, where the concern is governance and accountability.

When staff paste secrets into chatbots

The most common AI risk is the least dramatic: a well-meaning employee pastes confidential pricing, or an HR case with health details, into a free public chatbot under terms nobody has read. What happens to the data depends on the tool, and most leaders cannot say which tools their people use. This is ordinary data leakage made more tempting.

Outright bans tend to drive use onto personal phones and out of sight. The book recommends three leadership choices: provide an approved, business-grade tool with sensible data protections; define a short list of information that must never go into any AI tool; and decide how much monitoring is appropriate, since it touches employee privacy. At the fictional Meridian Foods, staff had visited forty-one different AI websites in one month. The risk committee approved a business licence costing a little less than the company's annual coffee spend, a one-page policy, and blocking of unknown services after a three-month grace period.

Deepfakes and why process beats perception

AI has removed many of the tell-tale signs of fraud. Scam emails can now be fluent in any language and written in an executive's own style, and a few seconds of recorded speech can be enough to imitate a voice. Detection tools exist but cannot be the main defence. The stronger answer is procedural: if unusual payments and sensitive changes always need confirmation through a separate, known channel, such as a call back to a number already on file or approval by two named people in the finance system, the quality of the fake matters much less.

Leaders decide which actions need this second check, typically new payees, changes to supplier bank details, payments above a threshold and payroll changes. They also remove the social risk for junior staff by saying publicly that anyone who checks a request from a senior leader will be thanked, not criticised.

An acceptable-use policy people will follow

Long, legalistic policies get signed and forgotten, and policies that read like lists of punishments make staff hide how they use AI. A useful policy fits on one or two pages and answers five questions: which tools are approved and how to request others; what must never go in; who is responsible for the output, since a person must check AI-drafted work and fluent but wrong answers are common; when AI use must be disclosed; and where to go with questions or mistakes, with a clear request to report within the hour.

Meridian's first draft ran to eleven pages; the second, written by a small mixed group, was a page and a half. The chief executive launched it by admitting he had almost pasted unannounced results into the approved tool. Three months later, use of unapproved AI sites had fallen by about seventy percent, and two people had reported their own mistakes, which the company treated as successes. The policy should carry a review date and a named owner.

Governing the AI you build or buy, with proportion

A customer chatbot can be tricked into promising refunds or giving wrong allergen information; a vendor forecasting tool raises questions about data use and who notices bad recommendations; a recruitment tool carries legal weight, as several jurisdictions treat AI in hiring or lending as high risk; and coding assistants must not become a reason to skip normal review. The book recommends a single register of AI uses with a named owner for each, review sorted by impact, legal, privacy and security review plus hostile testing for high-impact uses, and contracts that cover data use and training rights. For specialists it points to the NIST AI Risk Management Framework, ISO/IEC 42001 and the EU AI Act.

The chapter closes on proportion. AI also helps defenders sort alerts and spot unusual patterns, and many AI risks are old risks in new clothes. Meridian's board found two questions enough for any AI proposal: what is the benefit, and what is the worst realistic outcome and who is accountable for preventing it.

KEY POINTS

Key points

  • Find out which AI tools staff actually use today, approved or not.
  • Provide an approved AI tool good enough that people prefer it to free alternatives.
  • Publish a short list of information that must never be entered into any AI tool.
  • Require call-back verification through a known channel for unusual payments and bank-detail changes.
  • Keep an AI use register with a named owner and impact-based review for each entry.
  • Put a review date and an owner on the AI acceptable-use policy.

QUESTIONS

Common questions

Should we ban ChatGPT and other AI tools at work?
A ban usually fails because staff who find a tool that saves them hours keep using it on personal devices, out of sight. A more effective approach is to provide an approved business-grade AI tool with sensible data protections, publish a short list of information that must never be pasted into any AI tool, and keep visibility of which services are in use.
How can we protect the company against deepfake voice and video fraud?
Rely on process rather than on spotting fakes. Require every unusual payment, new payee, bank-detail change or payroll change to be confirmed through a separate, known channel, such as calling back a number already on file or approval by two named people in the finance system. Leaders should also say publicly that staff will be thanked for checking requests that appear to come from them.
What should an AI acceptable-use policy include?
A good AI policy fits on one or two pages and answers five questions: which tools are approved and how to request new ones, what information must never be entered, who is responsible for checking AI output, when AI use must be disclosed, and where to report questions or mistakes. It should encourage people to report errors within the hour and carry a review date and a named owner.
How should a board govern AI used in products and decisions?
Keep a single register of AI uses with a named business owner for each, and sort them by impact so that low-risk tools are not reviewed like high-risk ones. High-impact uses, such as screening job candidates or giving customers safety information, should get legal, privacy and security review and be tested by people trying to make them fail before launch. Vendor contracts should cover how your data is used and whether it trains their models.
What two questions should a board ask about any AI proposal?
First, what is the benefit in money, time or customer value? Second, what is the worst realistic outcome, and who is accountable for preventing it? If nobody can name the accountable person, the proposal is not ready for approval.