Files

Jake c95a5da077 docs(roadmap): bookmark SIE + adjacent embeddings/rerank/extract servers for PKM phase

Wyrdnote scales from voice-first dictation today to local-first
semantic PKM workbench tomorrow per the engine architecture spec.
The PKM phase needs a serving stack for embeddings + reranking +
entity extraction; this note bookmarks SIE (Superlinked Inference
Engine, Apache-2.0) as the trigger candidate plus six adjacents
to bake off against when the PKM phase lands.

Deferred because (1) PKM sits post-public-beta and infrastructure
decisions today would generalise poorly to a Q4 2026 selection;
(2) the candidates evolve fast; (3) Wyrdnote's local-first +
single-consumer-GPU constraint rules out a today-style bake-off
without real PKM-shape workload to test against.

Decision dimensions captured for the future evaluator: self-host
footprint, cold-start latency, throughput, quality on a Wyrdnote
eval set, reranker availability, extract/NER capability,
OEM-licensability against AGPL-3.0+dual-licence stack, telemetry
posture. Re-evaluation triggers documented.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-05-10 21:27:32 +01:00

5.2 KiB

Raw Blame History

name, description, type, tags, created, status, author, related

name

description

type

Wyrdnote PKM phase: tooling shortlist

The Wyrdnote engine architecture spec frames the product as a voice-first dictation app today, scaling to a local-first semantic PKM workbench tomorrow. The PKM scaling needs a serving stack for embeddings, reranking, and entity extraction. This note bookmarks the candidates worth evaluating when that phase lands.

Not in scope for the current roadmap (v0.1 dictation completeness, then the engine-architecture phases A through G in outputs/wyrdnote/2026-05-10-engine-architecture-spec.md). Pull this note off the shelf when PKM work begins.

Why deferred

PKM phase sits post-public-beta; pulling tooling decisions forward burns evaluation time on infrastructure that earns its keep months later.
The candidates below evolve fast; a 2026/05/10 bake-off would not generalise to a 2026/Q4 decision.
Wyrdnote's local-first stance means the chosen stack must run on a workstation with a single consumer GPU. Cloud-default tools are out, but BYO-cloud (user supplies endpoint) is acceptable as a configurable backend.

Primary candidate (the trigger for this note)

SIE — Superlinked Inference Engine. https://github.com/superlinked/sie. Apache-2.0. Open-source inference server bundling embeddings, reranking, and entity extraction behind three API functions (encode, score, extract). 85+ models hot-swappable, MTEB-verified in CI. OpenAI-compatible /v1/embeddings endpoint. Auto-detects CUDA / Apple Silicon / CPU. Ships a production stack (KEDA autoscaling, Grafana, Terraform for GKE/EKS, Helm chart) but the single-binary path is what matters for Wyrdnote.

Status as of bookmark: 25 commits, 1.7k stars, COMPATIBILITY.md acknowledging known rough edges. Worth re-checking maturity before selection.

Telemetry on by default; disable with SIE_TELEMETRY_DISABLED=1 or DO_NOT_TRACK=1. Wyrdnote's stewardship posture means this would be off in any embedded deployment.

Adjacent candidates to evaluate alongside SIE

When the PKM phase begins, run a head-to-head on a Wyrdnote-shaped corpus (real user note-lengths, mixed dictation + manual edits, British English vocabulary). Candidates worth pitting against SIE:

Ollama + a chosen embedding model (e.g. mxbai-embed-large, nomic-embed-text-v2, bge-m3). Already in Jake's wider plan for Smart Connections backend. Same single-binary character; no rerank or extract bundled.
text-embeddings-inference (HuggingFace, official). Apache-2.0. Embeddings + rerank only, no extract. Production-mature.
Infinity (michaelfeil/infinity). MIT. Embeddings + rerank + classification. Active community, strong perf benchmarks.
llama.cpp embeddings mode. The serving model is rougher but the inference engine is what Wyrdnote already uses for transcript cleanup; consolidating onto one inference runtime has real merit.
Voyage AI / Cohere embed + rerank as cloud BYO-API-key fallback for users who do not want to run a local model. Commercial; user-supplied key only.
Sentence-transformers as the lowest-rung baseline for the bake-off. Pure Python, well understood, runs anywhere. Sets the floor.

Decision dimensions for the future bake-off

When this note is pulled off the shelf, score each candidate on:

Self-host single-binary footprint. Disk + RAM + VRAM at idle with the chosen embedding model loaded.
Cold-start latency. First-encode-after-launch on the user's hardware. Pairs with the warmup coordinator from Phase D of the engine spec.
Throughput on Wyrdnote-shape corpus. Tokens / second, batch-size 32, 512-token sequences.
Quality on a Wyrdnote eval set. Build a small (~200 query) eval set from real notes; measure NDCG@10 and MRR.
Reranker availability. SIE bundles it; Ollama does not. The reranker matters more than the embedding model past a certain corpus size.
Extract / NER capability. Useful for auto-tagging notes, surfacing "people mentioned this week" views.
OEM-licensability. AGPL-3.0 + dual-licence OEM exception (≥£2k/yr) means we cannot wrap a GPL-only inference server. Apache-2.0 and MIT are clean. SIE is Apache-2.0; check the others.
Telemetry posture. Off by default or one-flag-off-able. Stewardship voice rules out anything that phones home with content.

Re-evaluation trigger

When ANY of:

Wyrdnote engine architecture spec Phases A through G are merged AND public beta has shipped.
A user-facing PKM feature lands on the roadmap (semantic search, "related notes" pane, auto-tag, knowledge-graph view).
The Ollama + Smart Connections route in Jake's wider stack hits a quality ceiling that SIE-class tooling could lift.

Until then: bookmark only.

5.2 KiB Raw Blame History