Nothing to Photograph

Every large firm now has a portfolio of AI pilots, and the portfolios are strangely alike. A document summarizer. An assistant that drafts the first version of a memo nobody reads carefully. A chatbot sitting on a policy library. A tool that turns a call transcript into a structured note. The demos work. The slides are honest. Eighteen months and a considerable sum later, the firm has changed nothing it would miss, and somebody senior concludes that the technology was oversold.

The technology was not oversold. The selection was broken, and in a way specific enough to name. These pilots were not chosen against pain. They were chosen against reviewability. Every one of them can be explained to a steering committee in eight minutes, measured inside a quarter, and failed in public without anybody losing a job. Almost nothing that actually costs the firm money meets all three conditions, which is exactly why almost nothing that costs the firm money is on the list.

There is a well-known five-step sequence for improving anything, and the order is the whole idea.

1. Question the requirement, and insist that it carry the name of a living person rather than the name of a department.
2. Delete the part or the process.
3. Simplify what survives.
4. Accelerate the cycle time.
5. Automate, last.

The warning attached to it is blunter than most management advice manages to be: the most common error of a capable engineer is to optimize a thing that should not exist.

It is usually read as a lesson about discipline, a reminder to be rigorous before being clever. That reading is too kind, because it locates the failure in the individual, and the failure is structural. Look at what each step produces that you can show someone.

• Question a requirement and, when it succeeds, you have a shorter list.
• Delete a process and you have an absence.
• Simplify and you have something that looks like the old thing with fewer moving parts, which photographs as almost nothing at all.
• Accelerate cycle time and you have a chart, and everybody in the room has learned to distrust that particular chart.
• Automate, and you have a screen. A live one. It moves, it produces output while people watch, it can be shown to the board, filmed, put in the annual letter and pointed at by the person who sponsored it.

Four of the five steps produce nothing to photograph. One produces a photograph. Firms fund photographs.

So the sequence does not merely get skipped through carelessness. It gets skipped by a mechanism that runs identically in every large organization, needs no bad actors, and looks like diligence from the inside all the way to the end. A capital process that requires demonstrable output before approval is a machine for jumping to step five, and no amount of quoting the correct order at people will change what that machine rewards.

The reviewability filter has three parts. It is worth being exact about what each one removes, because the pattern in what gets thrown away is the finding.

First: it must be explainable to a room. A pilot must be describable to eight people, most of whom do not do the work, in the time before the next agenda item. This immediately excludes every problem whose difficulty lives in the detail. Examples are the reconciliation that fails on nine unrelated edge cases, the handoff between two teams who describe the same object with different words, and the approval that is fast on paper but takes eleven days because only one person understands the exception. Those are the expensive problems. They are expensive precisely because they cannot be summarized, and the filter is a summarization test.

Second: it must be measurable inside a quarter. The clock is set by the reporting cycle rather than by the problem, so anything whose benefit arrives as the absence of a future event cannot be funded. Nobody gets promoted for the incident that did not happen or the reconciliation break that never opened. The pilots that survive this filter have a metric that moves while somebody is watching. That is a much smaller set than the set of things worth doing, and it correlates with value hardly at all.

Third: failure must be safe. This one does the most damage. A pilot must sit somewhere that being wrong is survivable, which sounds like prudence and works as a ban on touching anything that matters. The work where an error is unacceptable is, by definition, the work the firm considers important. Excluding it excludes the entire load-bearing structure of the business and leaves the periphery, and the periphery is where every one of those identical portfolios ends up.

Run those three filters together and you do not get a conservative selection of the valuable work. You get a systematically inverted one. The three tests are close to a negative image of importance, and a firm applying them faithfully will assemble, in total sincerity, a portfolio of impressive irrelevance.

There is a problem in field biology that describes this better than any case study.

Biodiversity records are collected by people who have to physically reach the place where the organism lives. The result, documented repeatedly and known in the literature as roadside bias, is that the great majority of specimen records in the global databases sit within a short walk of a road, a navigable river, a trail or a research station. Records cluster around field stations in particular, which is to say near the places where biologists already sleep. The published map of life on Earth is, to a first approximation, a map of access.

The important part is what this does to confidence. The bias does not show up as a gap, because the sampled areas are sampled thoroughly. The data are abundant, precise, repeated and rich in detail, and every one of those qualities makes the picture feel more trustworthy. A researcher looking at the records sees density and reads it as knowledge. The interior of the range, where nobody has walked, does not appear as a warning. It appears as nothing, and nothing is easy to mistake for empty. So the error compounds in the least helpful direction: the more effort goes into the accessible ground, the more authoritative the wrong map becomes.

Ecologists have known this long enough to build corrections. They weight for effort. They design surveys that deliberately sample the inconvenient interior. And when they publish a range map they publish the sampling effort alongside it, because a distribution without its collection history is not a finding. That discipline, not the observation of the bias, is the part that transfers.

A pilot portfolio is a specimen collection assembled by the same rule. It is dense, well documented and honestly recorded, and it maps the parts of the firm that were easy to reach. No firm publishes the sampling effort next to it. This is the old story of the drunk who searches for his keys under the streetlight, told properly, with the mechanism intact. The point of the story was never that the man was foolish for looking under the lamp. It is that he had a well-lit, thoroughly examined patch of ground and no reason, on the evidence in front of him, to believe the keys were anywhere else.

Now the part that makes this worse than waste, and it is the argument I would defend hardest.

A manual process carries its own cost, and that cost is not only a loss. It is a signal, and in most firms it is the only signal strong enough to eventually kill a bad process. Somebody has to do the thing every week, they resent it, they say so, the resentment accumulates, and one day the cost of the process exceeds somebody’s tolerance and the process is questioned and deleted. That is a slow and undignified mechanism. Where no one has the authority to think clearly about requirements, it is the mechanism that does most of the actual cleaning.

Automate that process and you have not removed it. You have removed the pain that would have removed it. The weekly reconciliation that four people hated is now free, silent and invisible, and it will therefore run until the company is sold. The approval chain nobody could defend now takes ninety seconds instead of nine days, which means nobody will ever again be angry enough to ask why it exists. The report that six people read has become a report that six people receive.

Automation is a preservative. It does not merely fail to delete the process. It actively makes the process immune to deletion, by removing exactly the friction that made the case for deleting it. Every firm that automated its way out of a bad approval chain has embalmed the chain and called it transformation, and the tell is that these things are never revisited. Nobody audits a cheap process. The cost that would have triggered the audit is the thing that was optimized away.

This is why the order of the five steps is not a stylistic preference. Running step five before steps one and two does not produce a worse version of the same outcome. It produces an outcome that is hard to reverse, because it turns a temporary bad decision into permanent low-cost infrastructure, and the firm’s evidence now argues for keeping it. The clearest public admission of this that I know came from Tesla in 2018, after its Model 3 lines, including the battery-module line in Nevada, were over-automated and had to be partly taken back out. The mistake was named as excessive automation, the ownership was personal, and the conclusion was that humans had been underrated. Taking automation back out is expensive, visible and humiliating. Firms will do almost anything to avoid that, which is exactly why the ordering matters more than the enthusiasm.

Here is the strongest objection to everything above. It is strong enough that I think it relocates the fault rather than answering it.

The five-step sequence assumes an engineer with authority over the requirement. Steps one and two are not techniques. They are exercises of power. Questioning a requirement means telling a named person that the thing they asked for is not worth what it costs. Deleting a process means overruling whoever put it there, who is often still employed and often senior. Neither step is available to someone without standing, and the person actually running an AI program in a large firm almost never has it. They can add. They cannot subtract. They can propose a new tool, and they cannot retire an old committee.

Given a person who may add but not subtract, jumping to step five is not indiscipline. It is the only legal move on the board. It also explains something otherwise puzzling: why these programs are so reliably staffed by capable people producing weak results. The constraint is not talent and not tooling. A firm that delegates the automation budget while keeping deletion authority several levels above it has guaranteed the outcome it is disappointed by, and has done so in the org chart rather than in any decision anyone remembers making.

The second objection is fairer still. Sometimes you genuinely cannot delete. The requirement is a regulation, the process is a legal obligation, the report is filed with a supervisor who will not be persuaded by an argument about elegance. In those cases automation is the correct answer, and the sequence agrees, because step one was to question the requirement, not to assume it is fake. The discipline is in demanding the name. A requirement with a person attached can be argued with, and sometimes the argument is honestly lost. A requirement attributed to “compliance,” or to “the way we do things,” or to a policy whose author has left, is not a requirement. It is a rumor with a budget.

One more concession, because starting where the light is has a legitimate use. A first pilot should absolutely be somewhere safe, legible and quick, because the purpose of a first pilot is to learn whether the team can ship anything at all. That is a training exercise, and it is defensible. What is indefensible is the second year of it. Firms never distinguish the two. The way to tell which one you are in is to ask whether anything since the first pilot has moved toward the difficult interior, or whether the program has simply produced more specimens from the same roadside.

The remedy is not a better prioritization framework. Frameworks are step three applied to step one, which is the same error in a different suit.

Name the person. Take the current list of pilots and, for each one, write down the name of the living person who asked for the underlying process to exist. Not the department. The person. Most of the list will fail at this line, and the failures are the finding, because a process nobody will claim is a process nobody is defending. Delete those first, and measure the program that year by what it removed rather than by what it shipped. A program with no deletions in twelve months has not been automating. It has been decorating.

Publish the sampling effort. Alongside the portfolio, write down which parts of the firm were never considered and why, in one line each, and read that list aloud in the same meeting. The list of exclusions is short to produce and almost never written, and it is the only document that distinguishes a deliberate strategy from a map of convenience. It will be uncomfortable, because the reasons will mostly turn out to be that the work was hard to explain, slow to measure, or dangerous to be wrong about, which is a compact description of the three things worth doing.

Accept a deletion that produces nothing to show. Fund it, count it, credit the person who did it, and let the steering committee sit through an agenda item with no screen attached. That single act is the whole reform, because everything else in this argument follows from a capital process that can only see things it can photograph. A firm that can approve an absence can run the five steps in order. A firm that cannot will run step five forever and call it momentum.

The keys are not under the lamp. They never were going to be. But the ground under the lamp is well lit, the search there is genuinely thorough, and everybody involved is working hard and telling the truth about what they found. That is why this particular failure is so hard to see from inside and so obvious from ten feet away. Turn the light off for an hour. Whatever you can still find is what you actually knew.