← Vibeset · case note
The 66,000-track stress test
Music identification demos look sharp on clean clips and collapse the minute real degradation hits them. We ran the opposite playbook — distortion as the default, not the edge case — and benchmarked it at catalog scale: 66,000 tracks, real distractor songs, no synthetic padding. The numbers are real, including the row where it fails.
01 — The problem
Repeated scanning, not demo theater
The economics of audio fingerprinting aren't set by peak accuracy on a leaderboard. They're set by scanning enormous catalogs over and over — every upload, every edit, every re-encode — without exploding latency or compute cost. A model that's slightly lower on a headline benchmark but dramatically cheaper and faster dominates real deployments. That constraint shaped every decision below.
02 — The system
Two speeds, one ear
Choon is tiered. A classical spectral-landmark pass — cheap, sub-second, deadly accurate on clean and filtered audio — handles the broad scan. When confidence drops, a compact neural pass takes over: a Conformer encoder distilled from a 300M+-parameter foundation teacher into a 27.7M-parameter student, serving embeddings from a FAISS index with temporal alignment on top. Each tier owns the failure modes the other can't: classical owns filters, neural owns pitch, tempo, reverb, and codec damage.
- audio in
- classical landmarks · fast pass
- neural embedding · robust pass
- match + alignment
03 — The wall
Small catalogs flatter everyone
Nobody buys the leaderboard. They buy the scan they can afford to run again tomorrow.
At a few thousand tracks, everything looks great. At 66,000 tracks with real distractor songs — not synthetic padding — our previous 48M-parameter model fell to 50% recall under attack. The fix wasn't a bigger model. It was better training against production-realistic distortions, plus a query-time calibration correcting a known pathology of high-dimensional retrieval (hubness — some references soak up matches they shouldn't). No retraining, no classical crutch: 50.0% → 76.9% overall, and 93.8% averaged over the core distortion conditions. With 42% fewer parameters.
04 — The bet
Small on purpose
This is my structured-pruning research thesis in production: capacity in large networks is misplaced, not missing. The shipped model is 27.7M parameters (110.6 MB) — 42% smaller than its predecessor and 26.9 points better at scale. Students as small as 12M beat the original at small scale. And the embeddings compress: at 32 dimensions the index shrinks 8× (67.6 → 8.4 MB) and search runs several times faster, for a recall cost of 0.2 points. Compression is free until 32 dimensions. Below that, it isn't — and we publish that cliff too.
76.9%
recall@1 overall, up from 50.0% — same catalog, same distractors
93.8%
averaged over the core distortion conditions
27.7M
parameters, 42% fewer than the model it replaced
66,000-track proxy catalog · real distractor songs · no synthetic padding
05 — What breaks
The conditions it still loses
- High-pass filtered audio — the neural tier's persistent blind spot; classical fusion exists because of it.
- 32 kbps MP3 — 128k and 64k hold above 99%; 32k collapses. Codec damage has a floor.
- Extreme pitch (±4 semitones) and heavy noise (SNR ≤ 5 dB) — near-zero, disclosed, and priced into the tiering.
- Embeddings below 32 dimensions — 8d retains 61% of variance and loses 30 points of recall. The cliff is real.
Every one of these numbers came from the same eval harness as the headline rows. The weak rows are the operational reality check — they're what tells you where the second tier, and the roadmap, have to live.
06 — Next
Provenance, not just recognition
Identification answers what is this? The other half is prove it — audio watermarking across multiple backends bound to C2PA-signed manifests, so a track carries its own provenance even after the internet chews on it. In active development; numbers when they're ready, same rules as above.