From c95a5da0771cdbd95953f5f1bb3353e1587168e9 Mon Sep 17 00:00:00 2001 From: Jake Date: Sun, 10 May 2026 21:27:32 +0100 Subject: [PATCH] docs(roadmap): bookmark SIE + adjacent embeddings/rerank/extract servers for PKM phase Wyrdnote scales from voice-first dictation today to local-first semantic PKM workbench tomorrow per the engine architecture spec. The PKM phase needs a serving stack for embeddings + reranking + entity extraction; this note bookmarks SIE (Superlinked Inference Engine, Apache-2.0) as the trigger candidate plus six adjacents to bake off against when the PKM phase lands. Deferred because (1) PKM sits post-public-beta and infrastructure decisions today would generalise poorly to a Q4 2026 selection; (2) the candidates evolve fast; (3) Wyrdnote's local-first + single-consumer-GPU constraint rules out a today-style bake-off without real PKM-shape workload to test against. Decision dimensions captured for the future evaluator: self-host footprint, cold-start latency, throughput, quality on a Wyrdnote eval set, reranker availability, extract/NER capability, OEM-licensability against AGPL-3.0+dual-licence stack, telemetry posture. Re-evaluation triggers documented. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../2026-05-10-pkm-phase-tooling-shortlist.md | 65 +++++++++++++++++++ 1 file changed, 65 insertions(+) create mode 100644 docs/roadmap/2026-05-10-pkm-phase-tooling-shortlist.md diff --git a/docs/roadmap/2026-05-10-pkm-phase-tooling-shortlist.md b/docs/roadmap/2026-05-10-pkm-phase-tooling-shortlist.md new file mode 100644 index 0000000..72053da --- /dev/null +++ b/docs/roadmap/2026-05-10-pkm-phase-tooling-shortlist.md @@ -0,0 +1,65 @@ +--- +name: Wyrdnote PKM phase — embeddings, reranking, extraction tooling shortlist +description: Bookmarked candidates to evaluate when the Wyrdnote PKM phase lands. Embedding model server + reranker + entity extractor for semantic search across notes. Not load-bearing for the dictation app; consumed by the PKM workbench layer that scales the product post-public-beta. +type: roadmap +tags: [wyrdnote, pkm, embeddings, reranker, tooling, shortlist, deferred] +created: 2026/05/10 +status: deferred +author: Wren on behalf of Jake Sames +related: + - outputs/wyrdnote/2026-05-10-engine-architecture-spec.md (CORBEL-Main) +--- + +# Wyrdnote PKM phase: tooling shortlist + +The Wyrdnote engine architecture spec frames the product as a voice-first dictation app today, scaling to a local-first semantic PKM workbench tomorrow. The PKM scaling needs a serving stack for embeddings, reranking, and entity extraction. This note bookmarks the candidates worth evaluating when that phase lands. + +Not in scope for the current roadmap (v0.1 dictation completeness, then the engine-architecture phases A through G in `outputs/wyrdnote/2026-05-10-engine-architecture-spec.md`). Pull this note off the shelf when PKM work begins. + +## Why deferred + +- PKM phase sits post-public-beta; pulling tooling decisions forward burns evaluation time on infrastructure that earns its keep months later. +- The candidates below evolve fast; a 2026/05/10 bake-off would not generalise to a 2026/Q4 decision. +- Wyrdnote's local-first stance means the chosen stack must run on a workstation with a single consumer GPU. Cloud-default tools are out, but BYO-cloud (user supplies endpoint) is acceptable as a configurable backend. + +## Primary candidate (the trigger for this note) + +**SIE — Superlinked Inference Engine.** `https://github.com/superlinked/sie`. Apache-2.0. Open-source inference server bundling embeddings, reranking, and entity extraction behind three API functions (`encode`, `score`, `extract`). 85+ models hot-swappable, MTEB-verified in CI. OpenAI-compatible `/v1/embeddings` endpoint. Auto-detects CUDA / Apple Silicon / CPU. Ships a production stack (KEDA autoscaling, Grafana, Terraform for GKE/EKS, Helm chart) but the single-binary path is what matters for Wyrdnote. + +Status as of bookmark: 25 commits, 1.7k stars, COMPATIBILITY.md acknowledging known rough edges. Worth re-checking maturity before selection. + +Telemetry on by default; disable with `SIE_TELEMETRY_DISABLED=1` or `DO_NOT_TRACK=1`. Wyrdnote's stewardship posture means this would be off in any embedded deployment. + +## Adjacent candidates to evaluate alongside SIE + +When the PKM phase begins, run a head-to-head on a Wyrdnote-shaped corpus (real user note-lengths, mixed dictation + manual edits, British English vocabulary). Candidates worth pitting against SIE: + +- **Ollama + a chosen embedding model** (e.g. `mxbai-embed-large`, `nomic-embed-text-v2`, `bge-m3`). Already in Jake's wider plan for Smart Connections backend. Same single-binary character; no rerank or extract bundled. +- **text-embeddings-inference** (HuggingFace, official). Apache-2.0. Embeddings + rerank only, no extract. Production-mature. +- **Infinity** (`michaelfeil/infinity`). MIT. Embeddings + rerank + classification. Active community, strong perf benchmarks. +- **llama.cpp embeddings mode**. The serving model is rougher but the inference engine is what Wyrdnote already uses for transcript cleanup; consolidating onto one inference runtime has real merit. +- **Voyage AI / Cohere embed + rerank** as cloud BYO-API-key fallback for users who do not want to run a local model. Commercial; user-supplied key only. +- **Sentence-transformers** as the lowest-rung baseline for the bake-off. Pure Python, well understood, runs anywhere. Sets the floor. + +## Decision dimensions for the future bake-off + +When this note is pulled off the shelf, score each candidate on: + +1. **Self-host single-binary footprint.** Disk + RAM + VRAM at idle with the chosen embedding model loaded. +2. **Cold-start latency.** First-encode-after-launch on the user's hardware. Pairs with the warmup coordinator from Phase D of the engine spec. +3. **Throughput on Wyrdnote-shape corpus.** Tokens / second, batch-size 32, 512-token sequences. +4. **Quality on a Wyrdnote eval set.** Build a small (~200 query) eval set from real notes; measure NDCG@10 and MRR. +5. **Reranker availability.** SIE bundles it; Ollama does not. The reranker matters more than the embedding model past a certain corpus size. +6. **Extract / NER capability.** Useful for auto-tagging notes, surfacing "people mentioned this week" views. +7. **OEM-licensability.** AGPL-3.0 + dual-licence OEM exception (≥£2k/yr) means we cannot wrap a GPL-only inference server. Apache-2.0 and MIT are clean. SIE is Apache-2.0; check the others. +8. **Telemetry posture.** Off by default or one-flag-off-able. Stewardship voice rules out anything that phones home with content. + +## Re-evaluation trigger + +When ANY of: + +- Wyrdnote engine architecture spec Phases A through G are merged AND public beta has shipped. +- A user-facing PKM feature lands on the roadmap (semantic search, "related notes" pane, auto-tag, knowledge-graph view). +- The Ollama + Smart Connections route in Jake's wider stack hits a quality ceiling that SIE-class tooling could lift. + +Until then: bookmark only.