The model is rarely the hard part. Across our production deployments, the gap between a computer-vision system that works in a demo and one that holds up on a live line has almost nothing to do with model architecture and almost everything to do with the physical and organizational details around it.
The parts nobody demos
Lighting consistency is the single most underrated variable on an inspection line. A model trained on images captured under one lighting condition will degrade the moment ambient light shifts with the time of day, a bulb ages, or a shift change moves a cart in front of a window. Before touching model architecture, we spend real time on physical camera rigs, consistent lighting enclosures, and a fixed capture distance — because no amount of data augmentation fully substitutes for a controlled capture environment.
The second is labeling discipline. Defect categories that seem obvious to an engineer are frequently ambiguous to the line workers doing the actual labeling, and inconsistent labels quietly cap a model's achievable accuracy no matter how much you tune it afterward. Getting inspectors and the model team aligned on a shared, written definition of each defect class — before collecting a single training image — is unglamorous and non-negotiable.
Where deployments actually fail
Three failure modes show up repeatedly:
- Data drift. A supplier changes a material finish, or a new product variant enters the line, and the model's accuracy quietly drops without anyone noticing until a defect escapes downstream.
- False-positive fatigue. A system tuned for maximum recall that also flags too many good parts trains line workers to distrust and start overriding it — at which point it's providing negative value even if its raw accuracy numbers look fine on paper.
- Integration friction. A model that works perfectly in isolation but wasn't built with a real interface into the line's existing control and reject-sorting hardware ends up running as a parallel system nobody actually uses.
What "working" actually looks like
Across the production lines we've deployed on — running at 99.4% inspection accuracy in aggregate — the pattern that holds up isn't a single powerful model, it's a narrow scope, a human-in-the-loop rollout period, and a retraining loop that treats drift as expected rather than exceptional. We deliberately launch on one defect class or one segment of a line first, keep a human reviewer in the loop on borderline calls for the first several weeks, and only expand scope once the false-positive and false-negative rates are stable under real operating conditions — not just validation data.
The retraining loop matters as much as the initial model. We build in a standing process for flagged edge cases to route back into the training set on a fixed cadence, rather than treating the model as a one-time deliverable that ships and is forgotten.
The actual takeaway
If you're evaluating a computer-vision vendor or planning an internal build, the questions worth asking aren't about model accuracy on a benchmark dataset. They're about the lighting rig, the labeling protocol, the false-positive tolerance the line can actually absorb, and who owns the retraining loop six months after go-live.