The pilot that produces evidence instead of anecdote

You have chosen the department. The next mistake is assuming the pilot will speak for itself. It will not. Run it without structure and what you get back, however genuinely positive, is a manager's impression: people liked it, it saved time, everyone wants to keep going. That is a story, and a sceptical board member is right to discount a story, however sincerely it is told.

The difference between a pilot that changes minds and one that gets politely ignored is almost entirely decided before the pilot starts, not after.

What "anecdote" looks like, even when well-intentioned

It looks like a summary built from memory rather than a record: "it felt faster." It looks like screenshots of the best outputs, selected after the fact, with nothing to show what a typical output looked like. It looks like no defined finish line, so the pilot simply continues until someone asks for an update, at which point the answer is whatever mood the team happens to be in that week. None of this is dishonest. It is just not evidence, and treating it as evidence in front of a board is where credibility gets spent for nothing.

The four things a pilot needs before it starts, not after

The fix is not more enthusiasm or a longer pilot. It is four decisions made in advance, in writing, before the first task runs.

• A success metric defined before the pilot starts. Decide what counts as success now, while nobody knows the outcome yet. A metric chosen after seeing promising results is not a metric, it is a rationalisation.
• A real baseline, measured, not remembered. Time the current process, count the current error rate, before the new approach touches it. Without this, "faster" and "better" are opinions with no reference point.
• A fixed end date, agreed in advance. A pilot with no end date does not end. It quietly becomes normal practice, unmeasured, and the organisation never gets the evidence it needed to decide anything.
• A named person who reports the result honestly, wins or not. Someone has to own the report and be willing to say the pilot did not work, if that is what happened. A pilot that can only produce good news is not a pilot.

A pilot without a metric decided in advance produces a story. A pilot with one produces evidence.

The trade-off nobody wants to name

All four of these cost real time before anything visibly starts: writing the metric, measuring the baseline, agreeing an end date, naming an honest reporter. It feels like bureaucracy layered onto something that was supposed to be quick and informal. The honest answer is that this upfront cost is what the evidence is made of. Skip it to save a week at the start, and you pay for it later with a result nobody in the room believes, at exactly the moment you needed them to.

The practical takeaway

Before the pilot starts, write the success metric, the baseline, and the end date on one page; if that page cannot be produced in an afternoon, the pilot is not ready to start yet.

No external statistic cited; this article presents an internal pilot-design method rather than third-party evidence.