docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jars
@@ -0,0 +1,86 @@
|
||||
---
|
||||
name: Audio file decoding (symphonia)
|
||||
type: architecture-map-page
|
||||
slice: 03-audio-transcription
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# Audio file decoding (symphonia)
|
||||
|
||||
> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → Audio file decoding
|
||||
|
||||
**Plain English summary.** When the user imports an audio file, `decode_audio_file` reads it through `symphonia`, downmixes to mono, and returns f32 PCM at the file's native rate. `decode_and_resample` chains this with `resample_to_16khz` on a blocking task so the Tauri command stays async. A short `probe_audio_duration_secs` helper reads the duration without decoding the body, used for "is this longer than the import limit?" preflight.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Crate: `magnotia-audio`
|
||||
- Paths: `crates/audio/src/decode.rs` (283 LOC), `crates/audio/src/concurrency.rs` (19 LOC).
|
||||
- External deps: `symphonia 0.5` with features `mp3, aac, flac, pcm, vorbis, wav, ogg, isomp4`.
|
||||
- Internal callers (best effort, slice 2 reconciles): the import-file Tauri command and the file-mode transcription command call `decode_and_resample`. Standalone `decode_audio_file` is exposed for tests and any path that wants the native rate.
|
||||
|
||||
Public surface:
|
||||
|
||||
- `pub fn decode_audio_file(path: &Path) -> Result<AudioSamples>` — `crates/audio/src/decode.rs:23`.
|
||||
- `pub fn decode_audio_file_limited(path: &Path, max_duration_secs: Option<f64>) -> Result<AudioSamples>` — `crates/audio/src/decode.rs:27`.
|
||||
- `pub fn probe_audio_duration_secs(path: &Path) -> Result<Option<f64>>` — `crates/audio/src/decode.rs:43`.
|
||||
- `pub async fn decode_and_resample(path: &Path) -> Result<AudioSamples>` — `crates/audio/src/concurrency.rs:11`.
|
||||
|
||||
## What's in here
|
||||
|
||||
### `decode_audio_file` and friends
|
||||
|
||||
`decode_audio_file` (`decode.rs:23`) delegates to `decode_audio_file_limited(path, None)`. The `_limited` variant accepts a hard cap in seconds; once the per-track sample count crosses `(max_duration_secs * sample_rate).ceil()` it returns `MagnotiaError::AudioDecodeFailed("Audio is longer than the {:.0} minute import limit", ...)`. Used by the Tauri import command to enforce the configured maximum.
|
||||
|
||||
Both call into `decode_media_stream` (`decode.rs:77`), which is the actual decode loop and the seam tests inject a `MediaSource` into. Steps:
|
||||
|
||||
1. Probe the format (`symphonia::default::get_probe()`).
|
||||
2. Pick the default track; reject if there is none or the codec sample rate is `0`.
|
||||
3. Build a decoder via `symphonia::default::get_codecs()`.
|
||||
4. Loop:
|
||||
- `format.next_packet()` — `UnexpectedEof` is the natural EOF marker; any other error is propagated as `AudioDecodeFailed`. `ResetRequired` is treated as a discontinuity error.
|
||||
- Skip packets that don't belong to the chosen track.
|
||||
- Decode into a `SampleBuffer<f32>`.
|
||||
- Average channels into mono if `channels > 1`; otherwise extend the output verbatim.
|
||||
- Apply the duration cap if one was provided.
|
||||
5. Reject empty output (`No audio data decoded`) and return `AudioSamples::new(samples, sample_rate, 1)`.
|
||||
|
||||
### `probe_audio_duration_secs`
|
||||
|
||||
`decode.rs:43`. Re-runs the probe step only, reads `track.codec_params.n_frames` and `sample_rate`, returns `Some(seconds)` if both are available. Cheaper than a full decode for "is this file too long?" gating.
|
||||
|
||||
### `decode_and_resample`
|
||||
|
||||
`concurrency.rs:11`. Wraps `decode_audio_file` + `resample_to_16khz` in `tokio::task::spawn_blocking`. Used by every async caller that needs ready-to-transcribe samples. Maps `JoinError` to `MagnotiaError::AudioDecodeFailed("Task join error: {e}")`.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Path
|
||||
└─ File::open
|
||||
└─ MediaSourceStream
|
||||
└─ Hint(extension)
|
||||
└─ probe → format + codec_params
|
||||
└─ Decoder
|
||||
└─ next_packet loop
|
||||
├─ packet → decoded → SampleBuffer<f32>
|
||||
└─ if channels > 1: mean across channels (mono mixdown)
|
||||
→ samples (f32, native rate)
|
||||
```
|
||||
|
||||
`decode_and_resample` chains the above with `resample_to_16khz` so the caller gets `AudioSamples` already at 16 kHz.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **RB-09 regression (2026-04-22 review).** The previous loop did `Err(_) => break` and silently returned partial audio on a mid-stream I/O error. Current code distinguishes EOF, `ResetRequired`, and other errors. Test at `decode.rs:262` injects a flaky `MediaSource` to keep this honest.
|
||||
- **AAC and ISO MP4 are pulled in unconditionally** by the symphonia feature set. Patent-licence implications for AAC are flagged in the brief; this is not a per-build decision today.
|
||||
- **Channels are averaged, not mixed psychoacoustically.** Stereo to mono goes via mean of all channels per sample. Fine for ASR; lossy for any future task that needs spatial cues.
|
||||
- **Whole-file decode loads everything into memory.** The duration cap is the only guard. Long imports exceed RAM long before they exceed disk.
|
||||
- **`Hint` only carries extension.** A `.mp3` named `.bin` will fail to probe even if the bytes are valid MP3. Caller responsibility to keep extensions truthful.
|
||||
- **`decode_and_resample` returns the resampled output.** If a downstream caller wanted the original rate (for a non-ASR purpose), it must call `decode_audio_file` directly.
|
||||
|
||||
## See also
|
||||
|
||||
- [Audio resampling](audio-resampling.md) — `resample_to_16khz` is the second half of `decode_and_resample`.
|
||||
- [Audio WAV I/O](audio-wav-io.md) — `read_wav` is the WAV-only fast path that uses `hound` instead of symphonia.
|
||||
- `magnotia_core::types::AudioSamples` (slice 5) — the value type returned.
|
||||
- Tests at `decode.rs:175` (RB-09 regression among others).
|
||||
Reference in New Issue
Block a user