If Your AI Disappeared Tomorrow
AI may be solving expertise gaps while quietly creating a new dependency problem for CEOs.
AI can improve decision quality while simultaneously reducing organisational capability.
The greatest AI risk may not be bad decisions, but the gradual erosion of human judgement.
CEOs should measure dependency on AI systems with the same discipline used to measure dependency on key executives.
In the latest season of The Capture, a powerful AI system called Simon sits at the centre of critical government decision-making. Simon analyses information, evaluates options and recommends actions faster than any human team could. At first, the people around it remain firmly in control. They review its recommendations, challenge its conclusions and make the final decisions themselves.
Over time, however, something subtle happens.
Simon proves itself useful often enough that trust begins to replace scrutiny. Nobody formally hands over authority, but fewer and fewer decisions are made without Simon’s involvement. The system becomes so embedded in the organisation that its absence becomes almost unthinkable.
While the story is fictional, the leadership challenge is not.
Across every industry, organisations are embedding AI into hiring, forecasting, pricing, customer service, risk assessment and strategic planning. Most discussions about AI focus on capability: what the technology can do, how much productivity it can unlock and where costs can be removed. These are important questions, but they are not the only questions CEOs should be asking.
An equally important question is what happens to organisational capability when AI begins performing work that previously developed human judgement.
From Leadership Dependency To System Dependency
For decades, leadership teams have worked hard to eliminate dependency on key individuals. Many of us have spent years building management capability, developing future leaders, introducing succession plans and distributing decision-making authority throughout our organisations. The logic was simple. Organisations scale when knowledge, judgement and ownership become distributed. They struggle when expertise remains concentrated in a handful of people.
AI introduces a very different path to scale.
Rather than creating more capable decision-makers, organisations can increasingly rely on systems that provide expertise, analysis and recommendations on demand. Need a forecast? Ask the model. Need market analysis? Ask the model. Need a recommendation? Ask the model. In many cases the answer is faster, cheaper and often better than the one available through traditional processes.
The productivity gains are real.
The risk is that capability may no longer be growing at the same rate as dependency.
This distinction matters because capability and dependency are not opposites. It is entirely possible for an organisation to become more productive while simultaneously becoming more dependent. We have seen this pattern before in other forms of technology adoption. Systems become more sophisticated, outputs become more reliable and organisations become increasingly confident in their use. Eventually the technology becomes embedded so deeply into daily operations that few people remember how to perform the underlying activity without it.
When Efficiency Replaces Capability
Recruitment provides a useful example.
Many organisations now use AI-assisted tools to screen, rank and prioritise candidates before a hiring manager ever reviews an application. Recent legal action involving Workday has focused attention on the role AI plays in hiring decisions and whether organisations fully understand the consequences of algorithmic recommendations.
What interests me more is the capability question.
How many hiring managers today are evaluating candidates, and how many are evaluating AI-generated shortlists?
Those activities may appear similar, but they develop very different skills. Over time, fewer managers gain experience identifying talent independently because the machine performs the initial analysis. The process becomes more efficient, but efficiency and capability are not always the same thing.
A similar pattern is emerging in knowledge work. Reports emerged that a major KPMG AI-related report contained fabricated references and examples generated through AI-assisted research processes. The issue was not that AI produced inaccurate content. Every executive who uses these tools understands that errors occur. The more interesting issue was that the content appeared sufficiently credible to pass through professional review processes.
The failure was not technological.
It was organisational.
People reviewing the material appeared to place greater trust in the output than they would have if the same work had been produced through traditional means. Authority had subtly shifted from the reviewer to the system.
Neither example represents a failure of AI.
Both examples represent a failure to maintain human capability alongside AI capability.
The Aviation Warning
Other industries have wrestled with similar challenges for years.
Aviation is often cited as one of the most successful examples of automation improving safety and consistency. Modern aircraft are extraordinarily sophisticated, and automation has contributed significantly to reductions in operational risk. Yet aviation experts have also spent decades discussing automation dependency and skill degradation. The concern is not that automation is harmful. The concern is that skills deteriorate when they are rarely exercised.
Pilots who spend less time manually flying aircraft may become less proficient at handling unusual situations when automation is unavailable. Airlines have spent years redesigning training programs to ensure pilots maintain critical skills despite increasing levels of automation.
The same question is beginning to emerge across knowledge work.
If AI performs the analysis, who develops analytical capability?
If AI produces the recommendation, who develops judgement?
If AI writes the first draft, who develops communication capability?
These questions become increasingly important as AI adoption moves from experimentation into core operational workflows.
The Metric Nobody Is Measuring
This is where the discussion becomes particularly relevant for CEOs.
Most organisations currently measure AI success through productivity improvements, cycle-time reductions, automation rates and cost savings. Those metrics are sensible and necessary. They help justify investment and demonstrate operational impact.
What they do not reveal is whether organisational capability is becoming stronger or weaker as AI adoption expands.
A business may improve efficiency dramatically while simultaneously reducing the number of people capable of making high-quality decisions without technological assistance. Over time, that can create a new concentration risk. Historically, organisations worried about dependency on key executives. Today they may need to worry about dependency on key systems.
The risk is not that AI becomes intelligent enough to replace leadership.
The risk is that leaders and teams become so accustomed to AI-supported decision-making that they gradually lose confidence in operating without it.
Capability migrates from people into platforms, often so gradually that nobody notices it happening.
The AI Dependency Audit
A useful exercise for executive teams is remarkably simple.
Imagine every AI system in your organisation became unavailable for the next thirty days. No copilots, no AI-generated reports, no forecasting assistants and no automated analytical tools.
Now ask four questions:
Which decisions would slow down?
Which teams would struggle most?
Which leaders would become bottlenecks?
Which capabilities would suddenly become scarce?
The answers are often revealing.
Every dependency exposed by that exercise represents a capability that may no longer exist where you think it does. Just as leadership teams conduct succession planning for key executives, they may soon need to understand their succession risk for critical AI-supported capabilities.
Final Strategic Truth
For decades, organisations focused on reducing dependency on key individuals. That was the right challenge for the era.
The AI era introduces a new one.
How do we capture the benefits of machine intelligence without allowing human judgement to quietly erode?
Because the objective was never to remove people from decision-making.
The objective was to create more people capable of making good decisions.
The CEO Question
If every AI system in your organisation disappeared tomorrow, would your people still know how to think?
The answer may reveal whether you are building capability at scale, or quietly outsourcing it.