Solving Silent Label Drift in Speech AI & ASR Datasets: Announcing driftcheck-audio on PyPI
π Excited to announce the official release of driftcheck-audio (v0.1.0) on PyPI! If you train Speech-to-Text (ASR) models (Whisper, Conformer, Kaldi) or curate speech datasets, you've hit this silent evaluation killer:
π Excited to announce the official release of driftcheck-audio (v0.1.0) on PyPI!
If you train Speech-to-Text (ASR) models (Whisper, Conformer, Kaldi) or curate speech datasets, you've hit this silent evaluation killer:
π Label drift and misaligned timestamps.
Speech datasets pair audio recordings with timed text (SRT, WebVTT, Lhotse manifests, Audacity labels). When these timestamps slipβdue to frame-rate mismatches (25 vs 23.976 fps), edits, or sync errorsβmodels silently train on pure noise or the wrong words. No exception is raised; Word Error Rate (WER) degrades silently.
driftcheck-audio solves this: an open-source Python library and CLI that audits timed labels against speech audio with mathematical precisionβwithout needing a heavy speech model.
π What Does driftcheck-audio Check?
β±οΈ Constant Offsets: Finds whether labels are early or late by milliseconds or seconds and provides the exact time-shift value to correct them.
π Steady Drift: Detects offsets that grow over time (e.g. video frame-rate mix-ups), reports the drift rate (ms/min), and calculates the stretch multiplier.
β‘ Sudden Jumps: Identifies mid-stream timing discontinuities caused by inserted ads, cut audio, or spliced takes.
ποΈ Unlabelled Speech & Phantom Labels: Quantifies speech omitted from labels and flags labels mapped over dead silence.
π² Null Permutation Testing: Compares every measurement against shuffled null labels to eliminate false positives and coincidental alignments.
π Real-Data Benchmarks (LibriSpeech dev-clean)
Validated across multi-speaker recordings from LibriSpeech dev-clean:
β Undamaged Labels: 0 failures, median measured offset of 0.001 s (1 ms). β Planted Offsets: 6/6 (100%) recovered across Β±0.2s to +2.0s shifts (median error 10β39 ms). β Drift Detection: 100% caught on 1.004 stretch with slope accuracy of 0.014 s/min. β False Positive Rejection: 0/6 false alignments when evaluated against unrelated text. β Test Coverage: 110 / 110 unit tests passing.
β‘ Lightweight & Fast (Zero Heavy ML Bloat)
Requires no PyTorch, no Whisper weights, and zero GPU compute. Built purely on speech energy envelopes, cross-correlation, and NumPy/SciPy/SoundFile for instant, sub-second execution on any CPU or CI/CD runner.
π¦ Quick Start & Links
Install directly from PyPI: pip install driftcheck-audio
Run an automated audit on your dataset in 1 command: driftcheck check talks/ --out report/
Supports SRT, WebVTT, CSV, Audacity .tsv, and Lhotse .jsonl.gz supervisions!
π PyPI: https://pypi.org/project/driftcheck-audio/
π» GitHub: https://github.com/BoobalanArjunan/Driftcheck
If you curate ASR datasets or build speech pipelines, give it a spin! β
SpeechAI #ASR #AudioML #MachineLearning #DeepLearning #Whisper #SpeechRecognition #Python #OpenSource #DataQuality #AudioEngineering #AI
Originally published by Dev.to AI. Aggregated on AIWithGhost for educational purposes β full credit and traffic to the original publisher.