AI Transparency Is Not Enough
What do we mean by AI transparency? In the definitions emerging from policy, it is too narrow and we need to start thinking about transparency more broadly. This post is the second of three about practical AI governance, continuing from this first post.
The general direction of AI legislation is to make AI use more transparent and maintain a clear line of accountability for the content. The EU’s AI Act is now in force and while it mostly talks about the responsibilities of organisations making and deploying AI systems, it does also cover businesses using those systems (‘deployers’). In a very broad summary of the Act (opens in a new tab), deployers need to (a) follow the provider’s instructions in using the system, (b) ensure a sufficient level of AI literacy among staff involved in the operating of the AI system, and (c) inform individuals when interacting with the system (“unless it is obvious from the context”). That last point extends further to specific use cases involving media content and so-called ‘high risk’ systems being used to make decisions involving the public. The EU law now sets the baseline for transparency, but it is not governance.
Better, But More Is Needed
I believe transparency is the direction of travel and I agree it should be. But this only answers the question about whether some system/ content was generated by AI. It doesn’t cover any information about why it was generated and how. And critically, what level of human accountability was included in the process.
The next question after “Is this AI?” is therefore “Can I rely on this?”. A good direction to evolve for the legal frameworks is to focus on transparency, reliability and accountability. Imagine a scenario where you are using an AI system to plan a project. You need the evidence that the data used was accurate, that the system made clear the assumptions and risks, and someone accepts accountability for the output.
The Stack
When you are the deployer of the AI system, you must acknowledge that there are different layers within the governance of that system. Think of this as a stack that builds up as follows:
- Key principles to define what is not negotiable
- Structured knowledge (updated and signed off as accurate by someone in authority) on which the system relies
- Full context about risks, including compliance requirements
- A layer that maintains evidence and the audit trail
- A layer that seeks and retains human sign off where needed
In the previous post, I talked about taking a systems view of the complexity. I think this idea of a governance stack is my current preferred way to start breaking the complexity down. There is a trap though in how to interpret this. If you treat each stack as being rigid with well-defined boundaries, you create problems.
Problems to Avoid
Firstly, you start to confuse governance layers with orchestration. Orchestration helps you to decide how the work gets done, in what steps, context and tools needed, how to check the results. This is not governance and hopefully you can see that confusing the two leaves important gaps.
Secondly, in attempting to define boundaries between these layers, you open your system up to edge cases that you didn’t see coming. Software developers and QA testers over the decades have obsessed about edge cases and developed championship-level expertise and best practices to sniff these out. For good reason: they introduce bugs in the system and can be incredibly expensive to fix. I’m constantly reminded of this when I try to automate something simple, only to discover a world of complexity and edge cases hiding in plain sight. The layers appear logical and obvious but they leak because the real process has more variety than your model of it, and the leaks are where those edge cases live. So the stack is not five clean interfaces, but rather five concerns with feedback loops running between them.
The Challenge
Now that we have drawn a line between governance and orchestration (both needed, but serve different purposes), let’s recap the thread. AI Governance starts with transparency but develops into something much richer that allows our users and customers to rely on the output. They can rely on the output because there is evidence of principles, assumptions, data quality, context, decisions and sign off baked into the core of the interaction/ content.
This is no small task. Consider this common situation: you decide to automate a productivity hack to make your life easier. You begin prototyping in Claude Cowork. You create a skill. You run it a few times, but something is not quite right. You lose confidence in the reliability of the skill but you keep using it knowing you are the quality check if something goes wrong. All you have managed to do is move the problem around, and possibly even opened up the risk that it fails silently and is now actively working against you.
Now consider that this is exactly how many people are using AI currently in their workplaces and jobs!
The Call to Action
We need to work differently and I believe practical governance is the key to a stronger foundation and a much better risk-reward on your automation projects and AI builds. In the third and final post, I will go into more practical detail using what we have discussed so far and demonstrate how to design governance into the problem from the start.
(This is my original thinking, developed and produced with AI support from Claude Opus 5.)