Work · Vision
defect-inspector.
A few good photos and a new image go in. A score, a heatmap and the region that looks off come out.
The problem
A few good photos, and a mark that is easy to miss.
Imagine a short line of parts and a handful of photos of good ones. An inspector looks at each new piece for a scratch, a dent, a stain. There are not enough labelled defects to train a classifier, and a miss is easy when the mark is small.
The useful result is not a model that “sees defects.” It is a score and a heatmap that mark the region that looks off, fitted from a few good samples and no defect labels.
What we built
A score and a heatmap, fitted from good samples only.
The pipeline is five steps. Detection does not need a language model. A few good photos are fitted with PatchCore, implemented here from the paper (wide-resnet50 patches, a greedy coreset memory bank, nearest-neighbour distance). A new image is scored against that bank; a heatmap and a box mark the region that looks off. Optionally, a vision-language model can label the region. You get a score, an overlay PNG, and JSON or CSV.
A yes-or-no flag without a place on the image is a rumour. A score with a heatmap is a claim you can look at — from the box back to the scratch, and dismiss it if the box is wrong. That is the difference between a detector you still inspect by hand, and one that can actually help.
Results
35 of 36 defects flagged, on three surfaces.
-
35 / 36
Defects flagged
-
1.000–0.958
Image AUROC · 3 surfaces
-
0.04–0.05 s
Per image · detection only
On three surfaces — metal, fabric and stone — PatchCore with wide_resnet50_2 flagged 35 of 36 defects, with image AUROC 1.000, 1.000 and 0.958. Pixel AUROC was 0.993, 0.993 and 0.957. One good stone image was flagged (1/18 good test images). Fitting took 2.0–4.6 seconds on eight images; scoring took 0.04–0.05 seconds per image (detection only).
| Surface | Image AUROC | Pixel AUROC | Flagged defect | Flagged good | Top region | Fit (s) | s / image |
|---|---|---|---|---|---|---|---|
| metal | 1.000 | 0.993 | 12/12 | 0/6 | 12/12 | 4.6 | 0.04 |
| fabric | 1.000 | 0.993 | 12/12 | 0/6 | 12/12 | 2.0 | 0.05 |
| stone | 0.958 | 0.957 | 11/12 | 1/6 | 10/11 | 2.3 | 0.04 |
| All | 35/36 | 1/18 | 34/35 | 2.0–4.6 | 0.04–0.05 |
Methodology. Three surfaces, 8 good training images, 6 good and 12 defective test images per surface (36 defective, 18 good, one defect per defective image). A defect is flagged when its score is above a threshold taken from the good training images. “Top region” counts flagged defects whose highest-scoring region overlaps the ground-truth mask. Backbone timm/wide_resnet50_2.tv_in1k, 12 November 2026. Detection timings are in docs/eval-detection.md. The raw table is in docs/eval.md. The script is scripts/evaluate.py; the samples are in samples/.
Region labels, scored with Qwen3-VL-2B-Instruct on the Hugging Face backend, were right on 29 of 34 flagged regions that hit the mask (12/12 metal, 8/12 fabric, 9/10 stone). The most common mistake on this set is calling a dent a spot. Labels are a description to help review, not a classification you can rely on.
Run it yourself
Three commands, then the demo.
git clone https://github.com/ModelShifts/defect-inspector
cd defect-inspector && python -m venv .venv && source .venv/bin/activate
pip install -e ".[demo]" && defect-inspector serveOn a CPU-only machine, install the CPU build of PyTorch first. Detection needs no language model. Add ".[hf]" or ".[api]" for optional region descriptions. Python 3.11 or newer.