feat(A.3 #21): VadChunker trait + RMS backend (Silero deferred)
New crates/transcription/src/streaming/ module with: - VadChunker trait: Send-bound, object-safe, push/flush/reset/ next_sample_index. Same surface a future Silero backend will present, so live.rs wiring does not change when Silero drops in. - VadChunk type: (start_sample: u64, samples: Vec<f32>) for commit-policy sample-offset bookkeeping in #24. - RmsVadChunker: fallback backend the plan permits while the ort 2.0.0-rc.10 vs rc.12 ecosystem conflict blocks silero-vad-rust / voice_activity_detector. Tuned to match the existing evaluate_speech_gate behaviour (enter 0.003, exit 0.0014, 3-frame onset, 500 ms silence close, 2 s max chunk). Key behavioural properties, each backed by a test: - pure silence emits nothing - samples between exit and enter thresholds never trigger onset - a single loud frame does not start a chunk (sustained speech only) - sustained speech followed by silence emits exactly one chunk - hysteresis: a dip between enter and exit does not split a chunk - max_chunk_samples caps continuous speech (Whisper never fed > 2 s) - flush surfaces in-flight speech - flush on an idle chunker emits nothing - reset restores a clean state - emitted chunk start_sample includes the onset buffer (Whisper sees the speech attack, not post-onset audio) Open items tracked as follow-ups: 1. Silero backend via direct ort rc.12 bridge (Handy-style). Blocked on either ecosystem ort alignment or dedicated bridge session. 2. Integration into src-tauri/src/commands/live.rs. Deferred so threshold tuning can be validated against real microphone captures rather than synthetic constant-signal fixtures.
This commit is contained in:
72
crates/transcription/src/streaming/mod.rs
Normal file
72
crates/transcription/src/streaming/mod.rs
Normal file
@@ -0,0 +1,72 @@
|
||||
//! Streaming primitives for live capture: VAD-gated chunking,
|
||||
//! agreement-based commit policy, and bounded buffer management.
|
||||
//!
|
||||
//! These types are tested at the unit level. Integration into
|
||||
//! `src-tauri/src/commands/live.rs` lands in follow-up commits so
|
||||
//! threshold tuning can be validated against real microphone captures
|
||||
//! rather than synthetic fixtures (brief items #21, #24, #25).
|
||||
|
||||
pub mod rms_vad;
|
||||
|
||||
pub use rms_vad::RmsVadChunker;
|
||||
|
||||
/// A span of audio the VAD considers worth transcribing. `start_sample`
|
||||
/// is an absolute index into the stream the `VadChunker` has been fed
|
||||
/// since its last `reset`; `samples` is f32 PCM at the chunker's
|
||||
/// configured sample rate.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct VadChunk {
|
||||
pub start_sample: u64,
|
||||
pub samples: Vec<f32>,
|
||||
}
|
||||
|
||||
/// A streaming VAD-gated chunker.
|
||||
///
|
||||
/// Implementations accumulate incoming samples, decide whether the
|
||||
/// current segment is speech using a score + hysteresis (brief item
|
||||
/// #21), and emit `VadChunk`s when a speech region ends — or when an
|
||||
/// in-progress speech region exceeds the configured max length so
|
||||
/// Whisper is not fed a 30-second monolith.
|
||||
///
|
||||
/// `push` returns any chunks ready to dispatch; typical usage is
|
||||
/// `for chunk in chunker.push(&samples) { dispatch(chunk); }` inside
|
||||
/// the capture loop.
|
||||
///
|
||||
/// `flush` is called at end-of-session to emit any in-flight speech as
|
||||
/// a final chunk (even if silence hasn't closed it).
|
||||
///
|
||||
/// `Send` because a chunker is owned by the live-session worker thread
|
||||
/// and moved into `spawn_blocking`.
|
||||
pub trait VadChunker: Send {
|
||||
/// Feed new samples. Returns any chunks the chunker has decided to
|
||||
/// emit as a result. An empty Vec means "still gathering".
|
||||
fn push(&mut self, samples: &[f32]) -> Vec<VadChunk>;
|
||||
|
||||
/// End-of-session: emit any in-progress speech as a chunk even
|
||||
/// though silence has not closed it. Returns `None` if there is
|
||||
/// nothing buffered (or only sub-threshold samples).
|
||||
fn flush(&mut self) -> Option<VadChunk>;
|
||||
|
||||
/// Drop accumulated state. Used between sessions on the same
|
||||
/// chunker instance (or after a context-window reset from the
|
||||
/// repetition detector — brief item #26).
|
||||
fn reset(&mut self);
|
||||
|
||||
/// Absolute sample index of the next sample that will be fed via
|
||||
/// `push`. Exposed so the commit policy (#24) can compute sample
|
||||
/// offsets for its agreement window.
|
||||
fn next_sample_index(&self) -> u64;
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn vad_chunker_trait_is_object_safe() {
|
||||
// Compile-time witness: keep the trait dyn-compatible so the
|
||||
// live-session worker can hold `Box<dyn VadChunker>` and swap
|
||||
// between RMS and Silero backends at runtime.
|
||||
let _: Option<Box<dyn VadChunker>> = None;
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user