Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
97 lines
4.1 KiB
Markdown
97 lines
4.1 KiB
Markdown
---
|
|
name: PCM bridge (AudioWorklet)
|
|
type: architecture-map-page
|
|
slice: 03-audio-transcription
|
|
last_verified: 2026/05/09
|
|
---
|
|
|
|
# PCM bridge (AudioWorklet)
|
|
|
|
> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → PCM bridge
|
|
|
|
**Plain English summary.** `static/pcm-processor.js` is a tiny browser-side AudioWorklet. It runs inside the Svelte frontend's `AudioContext`, naively downsamples the device's native rate to 16 kHz, and posts ~0.5 s batches of f32 samples back to the main thread. It only matters on a code path that goes browser-mic → frontend → Tauri command rather than the native cpal capture; today's primary path is `MicrophoneCapture` (Rust). Treat this file as a minimal fallback / web-context stub.
|
|
|
|
## At a glance
|
|
|
|
- Runtime: browser AudioWorkletProcessor (not a Rust crate).
|
|
- Path: `static/pcm-processor.js`
|
|
- LOC: 44
|
|
- External deps: none (Web Audio API).
|
|
- Internal callers (best effort, slice 1 reconciles): the Svelte frontend instantiates it with `audioContext.audioWorklet.addModule("/pcm-processor.js")` and listens for `port.message` events. The native cpal path in `magnotia-audio` is the production path; this exists for parity in dev/web contexts.
|
|
|
|
Public surface: registers the AudioWorklet processor name `pcm-processor`.
|
|
|
|
## What's in here
|
|
|
|
The whole file:
|
|
|
|
```js
|
|
class PcmProcessor extends AudioWorkletProcessor {
|
|
constructor() {
|
|
super();
|
|
this.buffer = [];
|
|
this.ratio = sampleRate / 16000;
|
|
this.needsResample = Math.abs(this.ratio - 1.0) > 0.01;
|
|
this.resamplePos = 0;
|
|
}
|
|
|
|
process(inputs) {
|
|
const input = inputs[0];
|
|
if (!input || input.length === 0) return true;
|
|
const samples = input[0]; // First channel (mono)
|
|
if (!samples) return true;
|
|
|
|
if (this.needsResample) {
|
|
// Simple downsampling to 16kHz
|
|
for (let i = 0; i < samples.length; i++) {
|
|
this.resamplePos += 1;
|
|
if (this.resamplePos >= this.ratio) {
|
|
this.buffer.push(samples[i]);
|
|
this.resamplePos -= this.ratio;
|
|
}
|
|
}
|
|
} else {
|
|
for (let i = 0; i < samples.length; i++) {
|
|
this.buffer.push(samples[i]);
|
|
}
|
|
}
|
|
|
|
if (this.buffer.length >= 8000) {
|
|
this.port.postMessage({ type: "pcm", samples: this.buffer });
|
|
this.buffer = [];
|
|
}
|
|
return true;
|
|
}
|
|
}
|
|
registerProcessor("pcm-processor", PcmProcessor);
|
|
```
|
|
|
|
`sampleRate` is the AudioWorklet global (the context's rate). The file picks the first channel and ignores the rest.
|
|
|
|
## Data flow
|
|
|
|
```
|
|
mediaStream → MediaStreamAudioSourceNode → AudioWorkletNode("pcm-processor")
|
|
└─ process(inputs)
|
|
├─ if needsResample (ratio != 1): drop-sample downsample to 16 kHz
|
|
└─ else: passthrough
|
|
└─ buffer to ~0.5 s (8000 samples)
|
|
└─ postMessage({ type: "pcm", samples: f32[] })
|
|
└─ frontend receiver (slice 1) → Tauri command (slice 2)
|
|
```
|
|
|
|
## Watch-outs
|
|
|
|
- **Drop-sample downsampling, no anti-alias filter.** Above ~8 kHz the result will alias. For ASR this is rarely a problem because Whisper expects 16 kHz inputs (so anything above 8 kHz Nyquist is already discarded), but if anyone repurposes this for non-ASR work it will show.
|
|
- **Mono only.** Reads channel 0, ignores channels 1+.
|
|
- **`Math.abs(ratio - 1) > 0.01` decides passthrough.** A 16001 Hz context (rare but legal) would skip the resample loop.
|
|
- **`port.postMessage` clones the array.** No transferables in this implementation; large session lengths magnify the cost.
|
|
- **The native cpal path is the primary path.** This worklet matters when the frontend captures audio itself (web build, dev preview without the Tauri shell). Slice 1 will know which routes mount it.
|
|
- **Build artefacts mirror the source.** Copies appear under `build/` and `.svelte-kit/output/client/` after a build. The source-of-truth file is `static/pcm-processor.js`.
|
|
|
|
## See also
|
|
|
|
- [Audio capture pipeline](audio-capture-pipeline.md) — the production path using cpal, not this worklet.
|
|
- [Audio resampling](audio-resampling.md) — the (much higher quality) sinc resampler used on the native path.
|
|
- Frontend Audio plumbing (slice 1) — the consumer of `port.postMessage`.
|