A second supplier agreement can be signed in an afternoon. Discovering whether the business can actually use the second supplier takes longer. The distinction tends to become visible at an inconvenient time, when the first supplier is unavailable and somebody asks why the backup needs three weeks of integration.

This is the test I would apply to my own operating-advantage thesis. If control over work and AI supply matters, an organization should be able to demonstrate a consequential alternative. If the alternative exists only in an architecture diagram, the thesis has produced stationery rather than strategic freedom.

Start with the dependency you would regret

Choose a material but bounded task. It should be important enough that failure changes a decision and small enough that the experiment can be reversed. A charming demonstration on a trivial task establishes portability of the demonstration.

Write down what the current provider supplies beyond model output. It might retain task state, retrieve documents, call tools, enforce permissions, or provide a particular deployment arrangement. If the second model can answer the prompt but the surrounding service cannot move, the workload has not moved.

Keep representative inputs and acceptance cases fixed for the comparison. Then run the task through another external provider or controlled inference, with the relevant rights and permissions in place. Record what changed in the implementation, how much expert time the move required, and which cases could no longer be served.

That last category belongs in the result. Excluding it would turn an exit rehearsal into an advertisement for the parts that exited successfully.

Beware the cooperative sample

In its February 24, 2026 experiment-design update, METR describes a problem that a larger spreadsheet cannot cure:

Our study is thus systematically missing developers who have the most optimistic expectations about AI’s value.

We are Changing our Developer Productivity Experiment Design METR · 24 February 2026 Wider adoption of AI has made it more difficult to measure task-level productivity.

The researchers put a number on the selection problem: 30%–50% of surveyed developers reported withholding some tasks because they did not want to perform them without AI. They also reported participation changes, lower compensation, and difficulty measuring time when agents ran concurrently. They judged the new data an unreliable signal of the current productivity effect. The point is not that their number transfers to an enterprise migration; it is that missing work can be systematically different from observed work.

That caution also qualifies a result people may remember. METR’s July 2025 randomized study assigned 246 issues from 16 experienced open-source developers to AI-allowed or AI-disallowed conditions and found that the AI-allowed tasks took 19% longer. That was an early-2025 result in a particular setting, not a timeless verdict on coding agents. The later update shows why an honest measurement program has to revisit both its tools and its sample.

A portability trial has an analogous trap. Teams are likely to nominate work they understand and can already move. Difficult dependencies remain in production, where they continue to be strategically important and experimentally absent.

I would therefore retain an eligibility register. Which task classes did we consider? Which did we exclude, and why? A local route that handles a clearly defined slice reliably can be valuable. A report implying general independence from that slice is not.

The failure cases need deliberate attention: incomplete input, a slow tool, loss of the incumbent endpoint, a rejected answer, and recovery after an interrupted action. These are tests to design around the chosen workflow, not a universal checklist that guarantees resilience.

A migration trial distinguishes candidate, eligible and attempted work from accepted outcomes, retaining exclusion reasons and failures.
Conceptual sample boundaries, not measured sample sizes. Report excluded and unattempted work beside the results; narrowing areas do not represent proportions.

Decide what losing would teach you

Before the trial, specify which outcome would change the investment. Otherwise each result can be interpreted as support for what management wanted already.

If the alternative meets the quality and service requirement at a tolerable total cost, a limited production route may be justified. If it fails on quality, the right decision may be to keep the external frontier service. If migration consumes weeks of work because business rules exist only inside one supplier’s configuration, the immediate investment may belong in extracting those rules rather than buying hardware.

A high switching cost is information. It tells the enterprise where dependence has accumulated. It does not, by itself, tell it to switch: the incumbent’s benefit may still exceed the cost of remaining. The point is to make that decision while there is time to choose.

There should also be a stop condition. If the workload is too small, the alternative too weak, and the continuity benefit immaterial, stop funding the second route. Continuing a failed independence project can create precisely the burden the strategy was supposed to escape.

Options need maintenance

A successful rehearsal is dated evidence. Models change, task definitions expand, tools move, and people leave. The fallback that worked six months ago can quietly become a historical artifact.

The sensible maintenance frequency depends on how quickly the dependency changes and how costly its loss would be. Retest after material changes and keep the operating owner identifiable. Include that upkeep in the option’s cost. The enterprise is purchasing an ability, not collecting a certificate.

That leaves a sharper version of the week’s argument. Cheap intelligence creates an opportunity to redesign production. The advantage, where it exists, belongs to an organization that can turn that input into accepted outcomes, learn from the exceptions, and retain credible choices about supply. Each part must earn its place in the economics; none becomes valuable merely by appearing in the framework.

The board-level question is not how many providers are approved or how many GPUs are owned. It is which important decision the business can now make without asking its incumbent supplier for permission. A working exit, even for a bounded class of work, is an answer. A second logo on a slide is a pending question.

The next question is what keeping that option alive actually costs: idle capacity, repeated evaluations, repairs, and the expert time behind an apparently autonomous service. That accounting can reverse the choice even after the migration succeeds.