Files
Lumotia/docs/architecture-map/03-audio-transcription/audio-pcm-bridge.md
jars a1f3f3f134 docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:

  01-frontend (16)              Svelte/SvelteKit UI
  02-tauri-runtime (26)         src-tauri commands + lifecycle
  03-audio-transcription (16)   audio + transcription crates
  04-llm-formatting-mcp (19)    llm, ai-formatting, mcp, cloud
  05-core-storage-hotkey-build  core, storage, hotkey, workspace,
                          (26) CI, dev glue

Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.

Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.

Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-09 14:04:13 +01:00

97 lines
4.1 KiB
Markdown

---
name: PCM bridge (AudioWorklet)
type: architecture-map-page
slice: 03-audio-transcription
last_verified: 2026/05/09
---
# PCM bridge (AudioWorklet)
> **Where you are:** [Architecture map](../README.md) → [Audio + Transcription](README.md) → PCM bridge
**Plain English summary.** `static/pcm-processor.js` is a tiny browser-side AudioWorklet. It runs inside the Svelte frontend's `AudioContext`, naively downsamples the device's native rate to 16 kHz, and posts ~0.5 s batches of f32 samples back to the main thread. It only matters on a code path that goes browser-mic → frontend → Tauri command rather than the native cpal capture; today's primary path is `MicrophoneCapture` (Rust). Treat this file as a minimal fallback / web-context stub.
## At a glance
- Runtime: browser AudioWorkletProcessor (not a Rust crate).
- Path: `static/pcm-processor.js`
- LOC: 44
- External deps: none (Web Audio API).
- Internal callers (best effort, slice 1 reconciles): the Svelte frontend instantiates it with `audioContext.audioWorklet.addModule("/pcm-processor.js")` and listens for `port.message` events. The native cpal path in `magnotia-audio` is the production path; this exists for parity in dev/web contexts.
Public surface: registers the AudioWorklet processor name `pcm-processor`.
## What's in here
The whole file:
```js
class PcmProcessor extends AudioWorkletProcessor {
constructor() {
super();
this.buffer = [];
this.ratio = sampleRate / 16000;
this.needsResample = Math.abs(this.ratio - 1.0) > 0.01;
this.resamplePos = 0;
}
process(inputs) {
const input = inputs[0];
if (!input || input.length === 0) return true;
const samples = input[0]; // First channel (mono)
if (!samples) return true;
if (this.needsResample) {
// Simple downsampling to 16kHz
for (let i = 0; i < samples.length; i++) {
this.resamplePos += 1;
if (this.resamplePos >= this.ratio) {
this.buffer.push(samples[i]);
this.resamplePos -= this.ratio;
}
}
} else {
for (let i = 0; i < samples.length; i++) {
this.buffer.push(samples[i]);
}
}
if (this.buffer.length >= 8000) {
this.port.postMessage({ type: "pcm", samples: this.buffer });
this.buffer = [];
}
return true;
}
}
registerProcessor("pcm-processor", PcmProcessor);
```
`sampleRate` is the AudioWorklet global (the context's rate). The file picks the first channel and ignores the rest.
## Data flow
```
mediaStream → MediaStreamAudioSourceNode → AudioWorkletNode("pcm-processor")
└─ process(inputs)
├─ if needsResample (ratio != 1): drop-sample downsample to 16 kHz
└─ else: passthrough
└─ buffer to ~0.5 s (8000 samples)
└─ postMessage({ type: "pcm", samples: f32[] })
└─ frontend receiver (slice 1) → Tauri command (slice 2)
```
## Watch-outs
- **Drop-sample downsampling, no anti-alias filter.** Above ~8 kHz the result will alias. For ASR this is rarely a problem because Whisper expects 16 kHz inputs (so anything above 8 kHz Nyquist is already discarded), but if anyone repurposes this for non-ASR work it will show.
- **Mono only.** Reads channel 0, ignores channels 1+.
- **`Math.abs(ratio - 1) > 0.01` decides passthrough.** A 16001 Hz context (rare but legal) would skip the resample loop.
- **`port.postMessage` clones the array.** No transferables in this implementation; large session lengths magnify the cost.
- **The native cpal path is the primary path.** This worklet matters when the frontend captures audio itself (web build, dev preview without the Tauri shell). Slice 1 will know which routes mount it.
- **Build artefacts mirror the source.** Copies appear under `build/` and `.svelte-kit/output/client/` after a build. The source-of-truth file is `static/pcm-processor.js`.
## See also
- [Audio capture pipeline](audio-capture-pipeline.md) — the production path using cpal, not this worklet.
- [Audio resampling](audio-resampling.md) — the (much higher quality) sinc resampler used on the native path.
- Frontend Audio plumbing (slice 1) — the consumer of `port.postMessage`.