# defect-inspector

A few good photos and a new image go in. A score, a heatmap and the region that looks off come out.

[Vision](https://modelshifts.com/work/) · 12 November 2026 · v1 · Python · [MIT](https://opensource.org/license/mit) · [CI](https://github.com/ModelShifts/defect-inspector/actions/workflows/ci.yml)

## The problem

Imagine a short line of parts and a handful of photos of good ones. An inspector looks at each new piece for a scratch, a dent, a stain. There are not enough labelled defects to train a classifier, and a miss is easy when the mark is small.

The useful result is not a model that “sees defects.” It is a score and a heatmap that mark the region that looks off, fitted from a few good samples and no defect labels.

## What we built

The pipeline is five steps. Detection does not need a language model. A few good photos are fitted with PatchCore, implemented here from the paper (wide-resnet50 patches, a greedy coreset memory bank, nearest-neighbour distance). A new image is scored against that bank; a heatmap and a box mark the region that looks off. Optionally, a vision-language model can label the region. You get a score, an overlay PNG, and JSON or CSV.

![Pipeline: a few good photos are fitted with PatchCore; a new image is scored against that memory bank; a heatmap and a box mark the region that looks off; an optional vision-language model can label the region.](https://modelshifts.com/media/work/defect-inspector/pipeline.svg)

Detection runs locally. Region labels, if you ask for them, use a vision-language model — your API, or an open model from Hugging Face.

![Left: a metal surface with a scratch, and the PatchCore heatmap highlighting it. Right: the same region described by a vision-language model.](https://modelshifts.com/media/work/defect-inspector/before-after.webp)

Metal surface. Image AUROC 1.000, 12/12 defects flagged, 0/6 good images flagged. scripts/evaluate.py.

A yes-or-no flag without a place on the image is a rumour. A score with a heatmap is a claim you can look at — from the box back to the scratch, and dismiss it if the box is wrong. That is the difference between a detector you still inspect by hand, and one that can actually help.

## Results

On three surfaces — metal, fabric and stone — PatchCore with wide_resnet50_2 flagged 35 of 36 defects, with image AUROC 1.000, 1.000 and 0.958. Pixel AUROC was 0.993, 0.993 and 0.957. One good stone image was flagged (1/18 good test images). Fitting took 2.0–4.6 seconds on eight images; scoring took 0.04–0.05 seconds per image (detection only).

| Surface | Image AUROC | Pixel AUROC | Flagged defect | Flagged good | Top region | Fit (s) | s / image |
| --- | --- | --- | --- | --- | --- | --- | --- |
| metal | 1.000 | 0.993 | 12/12 | 0/6 | 12/12 | 4.6 | 0.04 |
| fabric | 1.000 | 0.993 | 12/12 | 0/6 | 12/12 | 2.0 | 0.05 |
| stone | 0.958 | 0.957 | 11/12 | 1/6 | 10/11 | 2.3 | 0.04 |
| **All** |  |  | **35/36** | **1/18** | **34/35** | **2.0–4.6** | **0.04–0.05** |

**Methodology.** Three surfaces, 8 good training images, 6 good and 12 defective test images per surface (36 defective, 18 good, one defect per defective image). A defect is flagged when its score is above a threshold taken from the good training images. “Top region” counts flagged defects whose highest-scoring region overlaps the ground-truth mask. Backbone timm/wide_resnet50_2.tv_in1k, 12 November 2026. Detection timings are in docs/eval-detection.md. The raw table is in [docs/eval.md](https://github.com/ModelShifts/defect-inspector/blob/main/docs/eval.md). The script is [scripts/evaluate.py](https://github.com/ModelShifts/defect-inspector/blob/main/scripts/evaluate.py); the samples are in [samples/](https://github.com/ModelShifts/defect-inspector/tree/main/samples).

Region labels, scored with Qwen3-VL-2B-Instruct on the Hugging Face backend, were right on 29 of 34 flagged regions that hit the mask (12/12 metal, 8/12 fabric, 9/10 stone). The most common mistake on this set is calling a dent a spot. Labels are a description to help review, not a classification you can rely on.

## Run it yourself

```bash
git clone https://github.com/ModelShifts/defect-inspector
cd defect-inspector && python -m venv .venv && source .venv/bin/activate
pip install -e ".[demo]" && defect-inspector serve
```

On a CPU-only machine, install the CPU build of PyTorch first. Detection needs no language model. Add ".[hf]" or ".[api]" for optional region descriptions. Python 3.11 or newer.

[View the code on GitHub](https://github.com/ModelShifts/defect-inspector)

[Work](https://modelshifts.com/work/) · [Home](https://modelshifts.com/)