The detector found nothing, and reported a clean run
A face detector returned 46 faces across 40 photographs and exited 0. At the threshold I had chosen it found nothing at all in eighty photographs of people. On failure modes where an empty result is indistinguishable from a correct one.
Reportedfaces found: 46 across 40 photos
True0 of 80 photographs of people, at the threshold I had chosen.
I was sorting a few hundred recovered photos into folders by who was in them. Detect the faces, embed them, cluster them. People solved this years ago and the code is short.
The first run finished and printed this:
faces found: 46 across 40 photos
no faces : 184Forty-six faces in two hundred and twenty-four photos. That looked fine to me. Most of the folder was memes, screenshots and forwarded pictures of documents, so I expected plenty of images with no face in them.
I nearly moved on. The number was believable, and a believable number is the easiest kind to stop questioning.
Checking the negatives
I looked twice out of habit. When something reports a low count, I look at what it threw away.
So I checked what shape the 184 rejected images were:
35 1200x1600 JPEG
23 1600x1200 JPEG
14 738x1600 PNG
7 200x114 WEBPFifty-eight of them were ordinary phone photos, portrait and landscape. That is what you get when someone points a phone at a person. Screenshots are tall and thin and memes are small and square, so those fifty-eight were almost certainly pictures of people.
The number that should not have been possible
I ran the same detector over eighty of the rejected images at three confidence thresholds:
| Score threshold | Photos where a face was found |
|---|---|
| 0.85 | 0 of 80 |
| 0.60 | 28 of 80 |
| 0.35 | 40 of 80 |
At the threshold I had picked, it found nothing at all in eighty photographs of people. And the program called that a finished run with no errors.
When I dropped the threshold to 0.50, the full set went from 46 faces to 129, and from 18 people to a list that matched who was actually in the pictures.
Why 0.85 was wrong
0.85 is a reasonable-looking number. You pick a default like that because you want precision, and because every example you have seen shows a detector giving 0.99 on a clear face looking straight at the camera.
That is where the trouble starts. Detector benchmarks are built from photos that are well lit, well framed and mostly front on. Photos out of a messaging app are dim and taken at an angle. Half a face sits at the edge of the frame, and two rounds of compression have thrown away the fine detail the model looks for.
The model still finds those faces, but with low confidence. 0.85 asks for a level of certainty that real phone photos rarely reach.
What actually bothers me
The threshold is easy to fix. What bothers me is this: a tool that missed every face and a tool that correctly found no faces print the same thing.
Both show a low number and exit with code zero. There is no error and no warning, so a reviewer has nothing to catch. If the folder really had been all memes, 46 would have been right, and I would have shipped it without ever knowing.
Anything that detects, filters or searches can fail this way, because "nothing found" is a normal answer for it to give. A crash tells you something went wrong. An empty result given with confidence tells you nothing.
What I do now
I check the rejected pile. That is where a detector's mistakes are. The accepted pile only shows the cases that worked.
I try a few values before trusting a default. Running three thresholds over eighty images took one script and about four minutes, which is far cheaper than being wrong.
I make the tool print what it rejected and what shape those files were, so the next person sees the clue I almost missed.
I also wrote the reason into the source, right above the constant, so the number explains itself:
# At 0.85 the detector found faces in ZERO of 80 real phone photos. At 0.6 it
# found them in 28 of the same 80. Phone pictures are noisy, badly lit and
# rarely front on, so the confidence bar has to sit low. Set it too high and the
# tool silently finds nothing while appearing to work perfectly, which is the
# worst failure mode available.
MIN_FACE_SCORE = 0.50Someone will change that line one day. When they do, they will know what it costs.
The tool is at github.com/A-e70/face-grouper, MIT licensed. It groups photos by person and deliberately does not guess anyone's gender.