docs: architecture map (initial 5-slice generation, 105 pages)
Five-slice navigable map of the entire codebase under
docs/architecture-map/. Each slice is a self-contained
breadcrumbed sub-tree:
01-frontend (16) Svelte/SvelteKit UI
02-tauri-runtime (26) src-tauri commands + lifecycle
03-audio-transcription (16) audio + transcription crates
04-llm-formatting-mcp (19) llm, ai-formatting, mcp, cloud
05-core-storage-hotkey-build core, storage, hotkey, workspace,
(26) CI, dev glue
Plus master README.md and data-flow-end-to-end.md tracing
audio bytes from microphone to FTS5 search to MCP read.
Generated by 5 parallel subagents on 2026/05/09 against
HEAD 3c47000. Each page has YAML frontmatter, file:line code
refs, sibling cross-links, plain-English summaries.
Aggregated debt surfaced (full lists in master README):
RB-08 macOS power assertion, schema head drift v14 vs v15,
VAD blocked on ort version conflict, streaming primitives
not wired into live.rs, no prompt versioning, MCP has no
auth, cloud-providers in-memory keystore, SettingsPage
2 484 LOC, commands/live.rs 1 737 LOC, dual theme system,
brand rename to Lumenote pending across the codebase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
jars
@@ -0,0 +1,115 @@
|
||||
---
|
||||
name: Static assets and pcm-processor.js worklet
|
||||
type: architecture-map-page
|
||||
slice: 05-core-storage-hotkey-build
|
||||
last_verified: 2026/05/09
|
||||
---
|
||||
|
||||
# Static assets and `pcm-processor.js` worklet
|
||||
|
||||
> **Where you are:** [Architecture map](../README.md) → [Core, Storage, Hotkey, Build](README.md) → Static assets and `pcm-processor.js` worklet
|
||||
|
||||
**Plain English summary.** The `static/` folder ships verbatim with the app. Most files are obvious assets (favicon, fonts, a single texture). The interesting one is `pcm-processor.js`, an `AudioWorkletProcessor` that downsamples microphone audio to the 16 kHz mono PCM that whisper.cpp / Parakeet expect. It runs in the browser's audio worklet thread, not the main JS thread.
|
||||
|
||||
## At a glance
|
||||
|
||||
- Folder: `static/` at the repo root.
|
||||
- Files: `favicon.png`, `svelte.svg`, `tauri.svg`, `vite.svg`, `pcm-processor.js`, `fonts/`, `textures/`.
|
||||
- Total: ~250 KB (mostly the woff2 font files).
|
||||
- Consumers: Vite copies the directory into the build output; the SvelteKit static adapter serves the result; the in-browser webview hosted by Tauri loads from there.
|
||||
|
||||
## `pcm-processor.js` — the audio worklet
|
||||
|
||||
```js
|
||||
class PcmProcessor extends AudioWorkletProcessor {
|
||||
constructor() {
|
||||
super();
|
||||
this.buffer = [];
|
||||
this.ratio = sampleRate / 16000;
|
||||
this.needsResample = Math.abs(this.ratio - 1.0) > 0.01;
|
||||
this.resamplePos = 0;
|
||||
}
|
||||
|
||||
process(inputs) {
|
||||
const input = inputs[0];
|
||||
if (!input || input.length === 0) return true;
|
||||
|
||||
const samples = input[0]; // mono channel
|
||||
if (!samples) return true;
|
||||
|
||||
if (this.needsResample) {
|
||||
// Simple downsampling to 16kHz
|
||||
for (let i = 0; i < samples.length; i++) {
|
||||
this.resamplePos += 1;
|
||||
if (this.resamplePos >= this.ratio) {
|
||||
this.buffer.push(samples[i]);
|
||||
this.resamplePos -= this.ratio;
|
||||
}
|
||||
}
|
||||
} else {
|
||||
for (let i = 0; i < samples.length; i++) {
|
||||
this.buffer.push(samples[i]);
|
||||
}
|
||||
}
|
||||
|
||||
// Send every ~0.5 seconds at 16 kHz = 8000 samples
|
||||
if (this.buffer.length >= 8000) {
|
||||
this.port.postMessage({ type: "pcm", samples: this.buffer });
|
||||
this.buffer = [];
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor("pcm-processor", PcmProcessor);
|
||||
```
|
||||
|
||||
### What it does
|
||||
|
||||
1. Picks up the device-native sample rate (e.g. 48 kHz) from the `sampleRate` global available inside an `AudioWorkletGlobalScope`.
|
||||
2. Computes a downsampling ratio against 16 kHz. If the device is already 16 kHz (rare), passes samples through.
|
||||
3. The downsampler is **dropwise**: it accumulates a fractional position and emits a sample whenever the position crosses an integer step. No anti-aliasing filter. Acceptable for speech; subtly hurts on music or fast transients but Magnotia is a speech app.
|
||||
4. Buffers up to 8 000 samples (≈ 0.5 s at 16 kHz).
|
||||
5. Posts `{ type: "pcm", samples: [...] }` to the AudioWorklet's port. The main thread bridges this onward to the Rust live-session command.
|
||||
|
||||
### Why this lives in `static/` not `src/`
|
||||
|
||||
Audio worklets must be loaded via `audioContext.audioWorklet.addModule(url)` and resolved as a URL, not as a JS module imported into the bundle. SvelteKit's static adapter serves `static/pcm-processor.js` at `/pcm-processor.js`, which the worklet loader can fetch.
|
||||
|
||||
If we moved this into `src/`, Vite would bundle it and break the worklet load.
|
||||
|
||||
### Cross-link
|
||||
|
||||
The 16 kHz target rate is also the value of `magnotia_core::constants::WHISPER_SAMPLE_RATE`. **The worklet hard-codes the literal `16000` rather than importing the Rust constant** because audio worklets cannot import from the bundle. A coordinated change requires editing both places. See [`core-constants.md`](core-constants.md) for the Rust side.
|
||||
|
||||
The 8 000-sample emit threshold is also the value of `magnotia_core::constants::MIN_CHUNK_SAMPLES`. Same hard-coding issue.
|
||||
|
||||
## Fonts
|
||||
|
||||
`static/fonts/`:
|
||||
|
||||
- `atkinson-hyperlegible-next.woff2` — primary text font; designed for accessibility and dyslexia.
|
||||
- `instrument-serif-italic.woff2` — display italic.
|
||||
- `jetbrains-mono.woff2` — monospace, used in code blocks and the diagnostic-report bundle.
|
||||
- `lexend-variable.woff2` — UI alternative.
|
||||
- `opendyslexic.woff2` — accessibility font, opt-in via preferences.
|
||||
|
||||
Loaded via `@font-face` in `src/app.css` (slice 1). Local-first; no Google Fonts CDN.
|
||||
|
||||
## Textures
|
||||
|
||||
`static/textures/grain.png` — the subtle film-grain overlay used on the home page hero. Slice 1 territory.
|
||||
|
||||
## Watch-outs
|
||||
|
||||
- **`pcm-processor.js` literally hard-codes `16000` and `8000`.** Two magic numbers that must move in lock-step with the Rust constants. If Magnotia ever supports a different speech-model sample rate, this file is one of the touch points. Worth a build-time substitution mechanism (Vite plugin) to inject the constants from a single source.
|
||||
- **The downsampler does no anti-aliasing.** A device sampling at 48 kHz has frequency content above 8 kHz that should be filtered out before decimation. The current dropwise downsampler aliases that content into the 0-8 kHz band. Whisper handles the artefacts well in practice but a one-pole lowpass would be a cheap accuracy gain. Tracked verbally; no doc yet.
|
||||
- **Bypasses Vite optimisation.** Files in `static/` are not minified or fingerprinted. Acceptable for a worklet (load is once per session); cache invalidation is a non-issue because Tauri loads from local disk.
|
||||
- **Browser permission gate.** AudioWorklets require a `MediaStream` and run inside the audio context. Tauri's webview honours `getUserMedia` so the mic permission UI just works on Linux / Windows / macOS.
|
||||
|
||||
## See also
|
||||
|
||||
- [Frontend build config](frontend-build-config.md) — `static/` is served by the static adapter.
|
||||
- [Constants module](core-constants.md) — `WHISPER_SAMPLE_RATE`, `MIN_CHUNK_SAMPLES`.
|
||||
- [Slice 3 audio + transcription](../03-audio-transcription/README.md) — Rust side of the worklet bridge.
|
||||
Reference in New Issue
Block a user