Tend and Flow

Designing Governance

There is a flood of AI policy masquerading as governance, and it does not survive contact with an actual AI agent. If you want to automate a process, you first have to understand that process, including the messy bits nobody wrote down. Stafford Beer worked out why this matters back in the 1970s, and his insight is uncomfortably relevant now: your controls have to be as complex as the thing they are controlling. A prompt and some context will not do it; practical governance starts with feedback loops and safe experimentation, not documents.

The flood of policy dressed up as governance

I have become troubled by how much AI governance turns out to be policy wearing a framework’s clothing. Something goes wrong, or nearly does, and the response is reliably the same: a new document, a committee, a tweaked RACI chart. Everyone relaxes. Governance has been done.

Except it hasn’t. Nobody opens the document again until the next incident forces them to. This is not laziness, the people writing these things mean well and work hard. It is a design failure, and it becomes considerably more expensive the moment you point an AI agent at a real business process.

You cannot automate what you do not understand

Here is the part that I think gets skipped almost universally. If your objective is to automate a process using AI agents, then you need to understand that process. Not the version in the process map. The version that actually runs, with its informal workarounds, its exception handling, and the quiet judgement someone applies on a Thursday afternoon that never made it into anyone’s job description.

Where that understanding is incomplete, or where the real-world context around the process is only half-grasped, you are not automating efficiently. You are automating poorly, or you are automating the wrong thing altogether. Both are worse than doing nothing, because now it happens at speed and at volume.

Stafford Beer

I first became properly acquainted with Stafford Beer’s work in cybernetics after reading Dan Davies’ The Unaccountability Machine. I would highly recommend the book.

Beer’s work exposes something profound about a great deal of received business theory. Much of it is premised on optimising the organisation, tightening it, trimming the slack, until it can no longer cope with any uncertainty or disruption at all. We do this because of a very human inability to hold in mind all the interconnectedness of the systems and processes an organisation depends on, both internal and external. We simplify because we must, then we mistake our simplification for the territory.

A good article connecting Beer’s model to AI specifically is Stafford Beer and AI as Variety Engineering by Nick Swanson. Two ideas from Beer do most of the useful work:

  • The purpose of a system is what it does, not what it says it does.
  • A system must be at least as complex as the thing it is trying to regulate.

Sit with that second one for a moment. It is the whole argument.

Why your prompt will not save you

This gives us a genuinely useful framing for how to approach automation with AI, and it cuts directly against the prevailing advice.

When we instruct agents to complete a task, there is a strong pull towards believing that ‘context’ and ‘prompt’ are everything, that with a sufficiently detailed brief the agent will behave. I do not think that holds. You will almost certainly not be able to specify your way around the complexity of a real business process, because you do not fully perceive that complexity in the first place. That is precisely Beer’s point. And the complexity you failed to capture does not politely disappear. It surfaces as failure, in many different and creative forms.

Think of it like handing a very fast, very literal new starter a written procedure and no supervisor. They will follow it exactly, including off the edge of every situation the procedure never anticipated.

So the question becomes: how do we use systems thinking to be better at automation?

Governance on paper, governance in practice

On this, I enjoyed an erudite and well-researched article by Daragh O Brien of Castlebridge, which describes the aftermath of a failure or breach with painful accuracy:

“a new policy document, a committee, a tweaked RACI chart. Everyone has a general sense of relief that ‘governance has been done.’ Nobody consults any of it again until the next crisis. You have what I call ‘governance on paper’ but not ‘governance in practice’.”

I would recommend reading his section on feedback loops:

“Test your governance by what it does, not what it says it does - go and look at actual data flows and actual decisions, not just the policy describing them. Put authority and responsibility in the same hands: if someone is accountable for data quality or model risk, they need the standing and the resources to intervene, not just the title.”

So where does this leave you?

You are a founder, a director or a manager. You want to use AI to automate the things in your organisation that deserve automating. Good. You should. Three things follow.

Firstly, recognise that this is about feedback loops and iterative development. If you or your organisation are not yet good at experimenting safely, prototyping, and learning as you go, that is where to start. Not with an agent. With the capability to notice when something is going wrong and correct it quickly.

Secondly, accept that you will not be able to control AI the way you used to control software. The old model, specify it fully, test it once, sign it off, does not fit a system whose behaviour varies with context.

Thirdly, and this is the crux, understanding how to design and read feedback becomes the critical skill. Not policy authorship. Feedback.

This is what I have started calling ‘practical AI governance’: governance you can actually run on a Tuesday, in a real organisation, with the people and time you actually have.

What comes next

Here is the thing I find exciting about all of this. Once you stop trying to control complexity by writing it down, and start building the loops that let you see what is really happening, agents stop being a leap of faith and start being something you can steer. That shift is available to a five-person business, not just to an enterprise with a governance function.

In my follow-on post later this month, I will dig into the difference between governance and orchestration, two things that get conflated constantly and cause a great deal of trouble when they are. And I will set out practical guidance on how to actually begin with agents inside a complex, messy, real organisation.

Which is the only kind there is.

(This is my original thinking, developed and produced with AI support from Claude Opus 5.)

← Back to Journal