References
-
Danila Potapov, Matthijs Douze, Zaid Harchaoui, Cordelia Schmid. Category-Specific Video Summarization. ECCV 2014. The paper that introduced KTS.
-
Zaid Harchaoui, Olivier Cappé. Retrospective Multiple Change-Point Estimation with Kernels. IEEE/SP SSP 2007. The kernel change-point framework KTS builds on.
-
Ke Zhang, Wei-Lun Chao, Fei Sha, Kristen Grauman. Video Summarization with Long Short-Term Memory. ECCV 2016. Popularized KTS as preprocessing for the TVSum / SumMe benchmarks.
Related ideas
- Dynamic programming for change-point detection: the same optimal-substructure
recurrence appears in PELT, segmented regression, and
ruptures-style libraries. - The kernel trick: writing variance/distance purely through a Gram matrix is the same move behind kernel PCA and kernel k-means.
Sound bites & audio segmentation (Chapters 11 to 15)
- Charles Truong, Laurent Oudre, Nicolas Vayatis. Selective review of offline
change point detection methods. Signal Processing, 2020. Survey behind the
ruptureslibrary; PELT and kernel change-point cost functions. - Hervé Bredin et al. pyannote.audio: neural building blocks for speaker diarization. ICASSP 2020. The open-source diarization toolkit.
- Alec Radford et al. Robust Speech Recognition via Large-Scale Weak Supervision (Whisper). 2022. Timestamped transcription for sentence-level cuts.
- Alexei Baevski et al. wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations. NeurIPS 2020. Learned audio embeddings as KTS features.
- librosa:
https://librosa.org, standard audio feature extraction (MFCCs, spectral features) referenced in Chapter 14.
This book's code
code/kts.py, the full implementation.code/demo.py, the synthetic-data demo.code/kts_video.py, one-file CLI tool for real videos.code/soundbites.py, sound-bite extraction lab; imports the KTS core and applies it to audio.
All depend only on NumPy (Matplotlib optional, for the demo plot).