By Abdullah Zubair, VP – Technology, BCE Global Tech
For decades, organisations have invested in technology to make work faster and more efficient. AI is the latest chapter in that story, and its systems can write, decide, and act with barely a human in the loop.
What’s different this time is what happens underneath. The red flags we used to count on to catch a fake, clumsy phrasing, a strange request, a tone that did not sound like the person, are fading fast. AI can now produce an email, a voice, or a digital identity that is hard to distinguish from the original. According to Gartner, 41% of CISOs have reported at least one social engineering incident involving a deepfake in the past year.
If anything can be generated instantly and made to look authentic, an organization can no longer decide who or what to trust just by looking at it. Trust has to be designed into the system, not read off its surface. The more capable the technology gets, the more deliberate verification needs to become.
Governance That Earns Trust
Most AI governance programs fail for a simple reason: they are written as policy documents when they should be written as controls that sit inside workflows. Five principles hold up in practice.
First, every model, copilot, and agent needs a named human owner before it goes into production, because accountability does not scale the way compute does. Second, inventory precedes policy. Non-human identities already outnumber human ones by roughly 40-to-1 in the median enterprise. You cannot govern a population you have never counted.
Third, scale controls to consequence, not to technology. A summarization bot and an agent with payment-initiation rights are both “AI”, and treating them identically is how governance fails. Fourth, design for evidence, not for assurance. Logs should record the initiating identity, the credential used, the tool invoked, and why access was permitted. Lastly, access should also be time-bound by default.
The Myths Worth Retiring
The assumption I hear most often in boardrooms is that people can still spot a fake. In practice, they usually cannot. What actually stops deepfake-driven fraud has less to do with alertness and more to do with structure: callback rules, dual authorization, and out-of-band confirmation before any payment change goes through.
Another common misconception is that internal AI is low risk. Research from SailPoint found that 80% of organisations report their AI agents have already acted beyond their intended scope, with 23% exposing credentials outright.
Then there is the belief that provenance equals proof. Content credentials get stripped constantly by screenshots, exports, and platform pipelines. Provenance helps. It just does not settle the question on its own.
The Accuracy Trap
The paradox keeping leaders up at night is simple: the better a system performs, the less anyone checks it. This is automation bias, and it gets stronger as reliability improves.
An agent that is 99.9% accurate and makes 50,000 decisions a day is still wrong 50 times a day, quietly, often with production privileges attached.
Judgment is a Design Choice
Judgment does not hold up on willpower. It holds up when it’s built into the workflow. That means putting scrutiny where the cost of being wrong is highest, money movement and identity changes especially, and letting the machine run everywhere else, monitored in aggregate.
Before my teams accept an AI-produced action, I ask them to run through three questions: What does it cost if we’re wrong, and who absorbs that cost? Can it be reversed, and how quickly? And can we confirm it independently, without going back through the same system?
However, none of that works if the culture punishes caution. Reward the person who slows a transaction down and turns out to be wrong, because if challenging the machine is career-limiting, nobody does it twice.
What a Trusted Enterprise Looks Like
By 2030, trust will be something an enterprise can demonstrate on demand rather than assert in a brochure.
First, verified identity for machines. Registering an agent with an owner, a purpose, and an expiry will feel as routine as onboarding an employee. Second, provenance as infrastructure. The gap to close is verification, not signing.
Third, verification rituals that survive a convincing lie. Pre-agreed challenge phrases for executive requests, unglamorous and undefeated. Finally, reconstruct ability as something boards can measure: the mean time to reconstruct an identity’s timeline during an incident.
Years of building networks and security programmes have demonstrated that trust can no longer be inferred simply from behaviour or context. Modern security architecture increasingly starts from the opposite premise: trust must be explicitly issued, bound to an owner, and continuously reverified.
Otherwise, what appears to be trust may simply be habit.