Count the corrections as well as the completed runs
An automation can show hundreds of successful runs while an office manager spends the afternoon fixing its output. A technical success counter does not tell you whether the business has less work.
The most useful automation pilot metrics include the work that remains: reviewing, correcting, reconciling, and supporting the new process. Compare that effort with the old workflow before approving a wider rollout.
This guide is for an operator who has already chosen a candidate workflow. If you are still choosing, start with a workflow audit before AI automation.
Write the decision before collecting the numbers
Give the pilot an owner and one primary business goal. For example: reduce the effort needed to prepare a complete customer order without increasing incorrect orders.
Then define the decision you will make at the end: expand, revise, keep manual, or stop. Choose acceptable error levels and operating limits with the people responsible for the work. There is no universal savings percentage that makes automation worthwhile.
Keep the initial boundary narrow. Specify which order types, input channels, staff, and downstream systems are included. If the pilot handles only straightforward cases, report results for those cases. Do not present them as performance across the entire business.
Build a baseline that resembles the pilot
Observe the current process over a period that includes ordinary work and known busy periods. Record completed items, active staff time, waiting time, corrections, and exceptional cases. Include work performed outside the main system, such as checking an inbox or asking a supervisor for missing information.
Use the same definitions in both periods. A completed item should mean a usable business result, not merely a received message. Compare similar case types and note changes in staffing, demand, or input quality. A quieter week can make an unchanged process appear faster.
Start with a simple tally sheet: item type, active minutes, corrections, and completion status. Include the busiest recurring day or shift, and explain that you are measuring the process rather than ranking employees. Note where observation or incomplete records may affect the baseline.
| Metric | What to count | Why it matters |
|---|---|---|
| Active handling time | All staff minutes per completed item | Shows remaining effort |
| Quality | Incorrect results by type and consequence | Prevents speed from hiding harm |
| Review rate | Items requiring staff review / eligible items | Shows the real supervision load |
| Adoption | Eligible items using the new path / all eligible items | Exposes work that bypasses the pilot |
| Completion time | Time from receipt to usable result | Captures queue delays |
| Backlog | Unfinished items and age of oldest item | Exposes hidden work accumulation |
| Operating cost | Software, model/API use, hosting, and support | Tests ongoing affordability |
Keep denominators visible. “Ten errors” is hard to interpret without total volume. Do not report zero errors as proof of zero risk; state the sample size and what was checked.
Use a simple effort calculation
The following figures are hypothetical planning inputs, not vendor prices or Somnio client results.
Suppose 400 comparable items take six minutes each to process manually. That is 2,400 minutes, or 40 staff hours.
During the pilot, the same 400 comparable items reach a usable result. Routine checking takes 400 minutes. Corrections take 200 minutes. Monitoring and reconciliation take another 120 minutes. Total remaining effort is 720 minutes, or 12 hours.
Net capacity recovered = 40 hours − 12 hours = 28 hours for the measured period.
Count each task once. If correction time is already included in measured handling time, do not add it again. Include supervision that shifts from one employee to another; moving work to a manager is not eliminating it.
To estimate the value of recovered capacity, multiply those hours by the hourly cost of employing the staff, including wages and applicable benefits and overhead. Label the result as a capacity valuation. For a cash-savings calculation, use expenses that will actually fall, then subtract recurring automation costs. Track setup, training, temporary parallel processing, and service disruption separately from steady-state costs. Include these transition costs when estimating payback.
Do not divide implementation cost by a negative or uncertain monthly benefit and call it a payback period. If the evidence is incomplete, report what remains unknown.
Compare estimates with measured results
Somnio’s food delivery CRM case study uses 50 daily orders and five to ten minutes of manual entry per order. Its published 960–1,920 annual hours figure is a rounded estimate based on four to eight hours per day over 240 working days. It is not an audited annual time-tracking total.
That estimate can illustrate the scale of a manual task. It cannot predict your pilot’s savings. Your result depends on volume, exceptions, remaining review, working days, and whether the recovered time can be used productively.
Use labels such as “observed during pilot,” “client-reported,” and “annual projection” in your own report. A projection should state its assumptions, particularly if it extends a short pilot across an entire year.
Check whether quality improved at the same time
Review a representative sample of completed work, not only items the system flags as uncertain. Otherwise, confident mistakes can pass unnoticed. Separate harmless formatting problems from errors that affect a customer, payment, or delivery.
If AI interprets free-form text, record the model and prompt version used. A change can alter quality or cost. Recheck the relevant cases before expanding the rollout. Ordinary rules may be enough for structured data; a pilot does not need AI to be valuable.
Ask staff what work disappeared and what new work appeared. Their answers often explain a metric that looks unexpectedly good or bad.
Make the expansion decision explicit
Expand only when the agreed quality, effort, and operating requirements are met and someone can support the workflow. Revise when a specific fix has a plausible path to meeting them. Keep the manual process when it is simpler or more economical. Stop the pilot if serious errors cannot be contained.
The final report can be short: scope, baseline, pilot period, volume, effort, quality, costs, limitations, and decision. Attach the underlying measurements so another manager can follow the calculation.
Somnio’s AI Opportunity & Implementation Sprint assesses up to three candidates and implements one qualifying workflow. If no candidate qualifies, the outcome is a workflow map, scorecard, and recommended roadmap rather than a forced automation. Bring a baseline and a few representative examples to make that assessment more useful.
