Most AI pilots fail before the tool is even chosen. Not because the tech is bad, but because someone in a corner office picked a workflow, handed it to a vendor, and told the team to "try it out for a month." The people who actually do the work found out on a Tuesday standup. By week two, they were quietly routing around it.

A 30-day pilot works when the people whose jobs it touches help design it, measure it, and decide what happens next. Here is how I run them.

Before day one: pick the right thing to pilot

The single biggest predictor of pilot success is workflow selection. Get this wrong and nothing else matters. Get it right and you have a lot of forgiveness for mistakes downstream.

What you want: a repetitive, high-volume task that a specific team does every day, where the current process is annoying to the people doing it, and where "done" is unambiguous. Insurance eligibility checks. Inbound lead qualification. After-hours appointment requests. Recall calls for patients who missed their six-month cleaning. Refund request triage. Vendor invoice coding.

What you do not want: a strategic-sounding project with fuzzy outputs, no clear owner, and three departments in the RACI. "Use AI to improve customer experience" is not a pilot. It is a slide.

Ask the frontline team one question: "What part of your week do you dread?" You will get a shortlist in ten minutes. Pick from that list. If the automation removes something they hate, you have already won half the buy-in fight.

Week 0: the setup week nobody schedules

Officially the pilot is 30 days. Practically you need about a week before that to do the boring work that determines whether the pilot means anything.

Four things to nail down:

  • A baseline. Measure the current process for one full week. How many tickets, calls, or claims? How long does each take? What is the error rate? How often does something fall through? If you cannot compare against a real number, you will end the pilot arguing about vibes.
  • A named owner on the team. Not the VP. The person who does the work, or their direct supervisor. They own the daily feedback loop with the vendor or internal builder.
  • Two or three success metrics, written down. Something like: time-per-task down 40%, zero missed appointments in the pilot cohort, first-response time under five minutes. Pick metrics the team already trusts.
  • An explicit kill criterion. What would make you shut this down on day 15? Write it down now, when nobody is emotionally invested.

This week is also when you tell the team what is happening, in plain language, with real specifics. "We are testing a tool that will draft insurance follow-up emails so you can review and send instead of writing from scratch. It will not send anything without you approving it. If it makes your job harder, we kill it. Sarah is running it and wants your feedback daily."

Week 1: shadow mode

Do not put the AI in the driver's seat on day one. Run it in shadow mode: it produces outputs, a human compares those outputs to what they would have done, and nothing goes to a customer or patient without review.

This does two things. It surfaces the actual failure modes (and there will be some you did not predict), and it gives the team a chance to build trust on their own timeline. When a front-desk lead sees the agent correctly handle 40 out of 50 appointment requests and gets to catch the 10 weird ones, they start to believe. When they get handed something at 100% autonomy on day one, they spend the whole month looking for reasons to distrust it.

Track the disagreements. Every time the human overrides the AI, log why. That log becomes the tuning list for week two.

Week 2: tune and expand scope carefully

By now you have real data on where the automation is strong and where it stumbles. Fix the top three failure patterns. Do not try to fix everything. You are not shipping a product, you are proving a workflow.

This is also the week to start letting the AI take limited real actions with tight guardrails. Maybe it now sends the confirmation text on its own, but escalation cases still route to a person. Maybe it books appointments in defined slots but flags anything outside normal hours. Widen the scope one notch at a time.

Hold a 20-minute standup with the pilot team twice this week. Not a status meeting. A "what broke, what surprised you, what do you want changed" meeting. Take notes visibly. Ship at least one change they asked for within 48 hours. Nothing builds buy-in faster than being heard and seeing something actually change.

Week 3: run at target scope

This is the week the pilot has to prove it. Full intended scope, minimal hand-holding, measuring against the baseline you took in week 0.

Two things I watch closely here. First, is the team's workload actually lighter, or has it just shifted to a different kind of work they also dislike? A review queue that never gets reviewed is worse than the old process. Second, what is happening to the edge cases the AI cannot handle? If they are piling up unattended, that is a real problem, not a rounding error.

Have the pilot owner write a one-paragraph update at the end of every day. Three sentences: what worked, what did not, what we are changing tomorrow. This is your paper trail for the go/no-go conversation.

Week 4: decide, and be honest about it

The last week is for the decision, not for more feature building. You want a clear answer to three questions:

  • Did we hit the metrics we defined in week 0?
  • Do the people doing the work want to keep using it?
  • What does it cost to run at 5x the current scope?

If the answer to the first two is yes, expand. If the answer to the first is yes but the second is no, stop and figure out why before you scale. Forcing adoption on a team that resents the tool is how you end up with $60K in annual software costs and a shadow spreadsheet everyone actually uses.

If the pilot failed, say so out loud. Write up what you learned. Pick a different workflow. The teams I have seen build real AI capability are the ones that treat killed pilots as normal, not as career risk. If every pilot ships, you are not being ambitious enough with what you try.

What buy-in actually looks like

You know you have buy-in when someone on the team asks you to expand the automation without being prompted. When the front-desk lead says "can it also handle the recall calls?" you have won. When people on other teams start asking how to get one for their workflow, you have a program, not a pilot.

You do not get there with a launch email and a training deck. You get there by picking a workflow the team hates, measuring honestly, shipping changes they asked for, and being willing to kill things that do not work. If you want help scoping a pilot like this or building the automation itself, talk to our team.

Frequently Asked Questions

How many people should be on the pilot team?

Small. Three to six people who actually do the work, plus one owner. Larger pilot groups slow the feedback loop and dilute accountability. You can widen the group in month two if the pilot proves out.

Should we build in-house or use a vendor for a 30-day pilot?

For a first pilot, use someone who has shipped the workflow before, whether that is a vendor, a contractor, or an internal team with experience. A 30-day pilot is not the moment to also learn a new platform, build your first prompt architecture, and figure out your evaluation framework at the same time.

What if the team is worried the AI will replace their jobs?

Address it directly on day one, with specifics. Which tasks does the automation take, which tasks does it not touch, and what does the team's day look like on the other side. Vague reassurance makes it worse. If the honest answer is that headcount will change, say that too. People handle hard news better than uncertainty.

How do we handle sensitive data during a pilot?

Decide before day one what data the tool can see, where it is stored, who has access, and what audit trail exists. For healthcare, that means BAAs in place before any PHI touches the system, and confirming specifics with your compliance counsel. Do not pilot on real sensitive data with a "we will figure out compliance later" plan. You will not figure it out later.

What is a realistic ROI expectation for a first pilot?

A well-scoped operational pilot usually shows a 30-60% time reduction on the targeted task within the 30 days. That is not the same as headcount savings, and you should not promise it as such to your CFO. The bigger win in month one is proving the pattern, so pilots two and three go faster and you build internal muscle for what "good" looks like.