S1 · E01
A conversation about lost context became the ConvoCanvas plan: preserve conversations, organise them in a vault and turn them into useful knowledge. The published episode describes planning here; implementation follows in the next chapter.
Chronology correction: this was the planning conversation. Vault implementation and ChromaDB indexing appear in the following September chapters.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
S1 · E02
The vault, LibreChat and a first FastAPI backend gave the idea a working path: upload a conversation, parse it and extract content ideas.
Historical account from the published journal. Dates describe engineering, not publication.
Obsidian VaultConversation MVP
Related decisions
S1 · E03
Ollama and local models opened a second path for working with conversations. The episode explores control, model selection and the limits that remained.
Historical account from the published journal. Dates describe engineering, not publication.
Ollama
Related decisions
S1 · E04
ChromaDB integration turned a growing collection of Markdown into semantic search. The episode follows the implementation and scaling work across the week.
Historical account from the published journal. Dates describe engineering, not publication.
ChromaDB
Related decisions
S1 · E05
Research into update speed and operational cost led to a hybrid deployment decision. Existing K3s workloads did not mean every service should move into the cluster.
Historical account from the published journal. Dates describe engineering, not publication.
K3s Cluster
Related decisions
K3s · design note →S1 · E06
A failing cluster exposed networking and recovery assumptions. The chapter follows diagnosis, recovery and the need for better operational visibility.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
K3s · design note →The story
FastSearch (<1s), DeepResearch (~10s), and AgentRouter with heuristic-based intent classification.
FastSearchDeepResearchAgentRouter
S1 · E07
Generated architecture diagrams made the growing platform easier to inspect and reduced the work of maintaining a separate mental picture.
Historical account from the published journal. Dates describe engineering, not publication.
Architecture Diagrams
Related decisions
S1 · E08
The same vault and search tooling helped reconstruct the first season. Publishing and episode navigation became part of the system being built.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
S2 · E01
Reconstructing the early journal exposed a date problem: archive filenames could obscure when the work actually happened. Temporal search and metadata handling were corrected.
Historical account from the published journal. Dates describe engineering, not publication.
Temporal Filter
Related decisions
S2 · E02
Code review, systematic checks and metadata debugging tested whether recent improvements behaved as expected. The story makes the verification work visible alongside implementation.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
S2 · E03
Capture cleanup, semantic chunking, architecture auditing and publishing tools advanced across several short sessions. The episode distinguishes active work from the elapsed overnight window.
Historical account from the published journal. Dates describe engineering, not publication.
GPU MonitorPerplexity GW
Related decisions
S2 · E04
A vault reindex, Journal v3, hybrid search and developer commands met in one testing window. Feature flags and measured trade-offs mattered as much as getting services deployed.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
S2 · E05
Agent patterns, evaluation and resumable work were introduced alongside recovery and learning work. The episode also records the gap between agent-assisted execution and personal understanding.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
The story
Model Context Protocol bridge connecting Claude Code to the platform via SSE. The interface that makes everything accessible from a conversation.
MCP Bridge
The story
First K3s manifests committed. Platform moving from systemd services to Kubernetes with FluxCD GitOps.
K3s ClusterFluxCD
The story
RAG Gateway, Retriever, MCP Bridge, Ollama, and ChromaDB all deployed to K3s. Prefect work queues wired. CrashLoopBackOff fixed (K8s env var parsing). Platform is cloud-native.
RAG GatewayRetrieverOllamaPrefect
The story
vLLM deployed on RTX 4080 for GPU-accelerated model serving. Ollama relegated to CPU-only fallback. 30x embedding speedup.
vLLM
The story
Program Dependency Graph deployed to production. OPA policy enforcement, DORA metrics exporter, Discord notifications for high-risk MRs. 221 passing tests.
PDG GeneratorOPA ServerDORA Metrics
S3 · E01
A long engineering arc brought storage migration, a rebrand and a more structured platform. This is a retrospective covering several months, not a single-day deployment.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
The story
spaCy NER (en_core_web_lg + EntityRuler + YAKE) replaces LLM-based tagging at ~1000x lower cost. ADR-040 accepted.
NER Consumer
The story
VW Jira project created. Jira-first development workflow, automated Confluence documentation, bidirectional vault-Confluence linking. LinkedIn automation pipeline (ADR-001).
JiraConfluence
S3 · E03
19 Bitnami Sealed Secrets deployed. Harbor container registry on K3s. NetBox IPAM for infrastructure tracking.
Historical account from the published journal. Dates describe engineering, not publication.
HarborSealed Secrets
Related decisions
S3 · E02
Unreliable generated journals prompted a Kafka-based pipeline, fuller session context and quality checks. The published chapter records iterations and limits rather than treating generated prose as evidence by itself.
Historical account from the published journal. Dates describe engineering, not publication.
KafkaSession Parser
Related decisions
The story
VW-51/52: Dead ChromaDB import causing 7.5GB per replica. 93% memory reduction. Kafka consumer crash-loop fixed (VW-62, 837 restarts).
The story
ChromaDB replaced by Qdrant (148K chunks). FlashRank reranker deployed, then replaced by ColBERT after benchmark comparison. FalkorDB knowledge graph activated (VW-79).
QdrantFalkorDBReranker
The story
Qwen3-Embedding-4B at 2560 dimensions. Graph ingestion bug fixes (VW-85/86/90 — asyncio event loop scoping). HLD/LLD modularised (ADR-016).
Embedding 2560d
The story
55+ NetworkPolicies deployed (default-deny + per-service whitelists). Vault indexer exclusion list alignment after 13,880-chunk data leak incident.
NetworkPolicies
The story
Full LinkedIn automation pipeline: text posts, PDF carousel generation (Puppeteer + pdf-lib), and publishing via REST API. ADR-019: Claude Voice Agent architecture proposed.
LinkedIn API
S3 · E04
The Matrix bot checked retrieval relevance before synthesis and refused when the evidence was too weak. The episode dates this engineering to February; the public ADR carries a later record date.
Historical account from the published journal. Dates describe engineering, not publication.
Refusal Gate
Related decisions
Refusal gate · design note →The story
Prometheus, Grafana (18+ panels), Tempo (distributed tracing), Loki (log aggregation), Beyla eBPF, Blackbox Exporter. SLOs defined: RAG Gateway 99.9%, MCP Bridge 99.5%.
PrometheusGrafanaTempoLoki
The story
Kafka event bus activated. File watcher (inotify) → Kafka → NER consumer (spaCy + YAKE) → auto-tagging → Qdrant indexing. KEDA scales consumers 0→N.
KafkaFile Watcher
S3 · E05
FalkorDB code_knowledge graph live: 7,768 nodes. Community detection (Leiden clustering), impact radius queries, tree-sitter AST parsing.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
ADR-020 · decision context →ADR-040 · decision context →The story
Biggest single-day push. Leiden clustering, TreeSitter, local model optimisations, Scout fleet deployment, shared_constants.py for exclusion lists, benchmark suite designed.
Scout Fleet
The story
VW-147 security hardening. Unleash feature flags deployed. CRAG and Speculative RAG prototyped behind flags. Infrastructure upgrades across the cluster.
Unleash
S3 · E06
A security investigation became a chain of dependency repairs, rollback, metadata cleanup and documentation work. The published event window is March, separately from the later SRE milestones.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
ADR-040 · decision context →S3 · E07
Benchmark-led fixes exposed scoring defects and a costly search path. The MCP transport migration improved connection behaviour, while deeper retrieval bottlenecks remained.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
Search benchmark · design note →MCP transport · design note →S3 · E08
An async retrieval refactor and a path-traversal fix improved different parts of the platform. The account also explains why retained journals had to stand in for missing raw transcripts.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
The story
ADR-034: Multi-source connector architecture. Jira, Confluence, GitLab, and vault connectors for unified indexing into Qdrant.
Connectors
The story
ADR-029: KEDA scales TEI embedding service 0→1 on demand, swapping GPU between vLLM and TEI. Desktop + MacBook multi-device setup via Tailscale.
KEDATEI Embedding
S3 · E09
Fast search gains did not solve deep-search latency. Graph freshness, incomplete consumers and connector design led to a more explicit retrieval and evaluation programme.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
ADR-034 · decision context →ADR-035 · decision context →ADR-036 · decision context →ADR-029 · decision context →The story
Apache Flink deployed for real-time conversation processing. Journal session parser, synthesizer, and DLQ reprocessor. Claim-check pattern for large payloads.
Apache FlinkSession Parser
The story
14-package uv workspace. 6-stage migration plan. Canonical module boundaries, per-package AGENTS.md, service Dockerfiles with workspace-dep linting. 40 commits.
uv Workspace
The story
Postgres-backed structured memory pipeline. GDPR-compliant tombstoning (ADR-043). 6 recall MCP tools: session, origin, decisions, state-at-time, trace chain, save context. Backfill flow processes 222 sessions. Git commit trailers (VW-393).
Memory SystemPostgres
The story
Empirical study: layered retrieval (typed→semantic→grep) scores 0.954. 5 conditions, 3 model tiers, 36 generated ADRs. Published as open-source on GitHub.
Benchmark Suite
The story
Autonomous SRE triage: Alertmanager webhook → event collector → LLM-powered brain → Jira issue creation. ADR-047 (LLM Content Trust Boundary), ADR-048 (Egress Control with Squid proxy).
SRE AgentAlertmanager
The story
Per-result vector_score, bm25_score, rerank_score. Dynamic Alpha Tuning (arXiv:2503.23013) — LLM-as-judge per-query weighting. Temporal edge timestamps on FalkorDB PDG. Competitive analysis: 5 research repos evaluated.
DAT Tuning
The story
Personal website deployed to Cloudflare Workers. Contribution heatmap (GitHub + GitLab), interactive architecture diagrams, CV from vault. Built with Astro 6, React Islands, and Tailwind CSS 4.
rduffy.uk
S3 · E10
The repository became a sixteen-package workspace while CI, memory evaluation and embedding contention were addressed. The chapter closes the season with clearer boundaries and a stronger decision trail.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
ADR-030 · decision context →ADR-054 · decision context →ADR-053 · decision context →S4 · E01
A structured audit found defects across client isolation, Kafka durability, feature flags and graph handling. The work paused and resumed; findings still needed remediation and verification.
Historical account from the published journal. Dates describe engineering, not publication.
Audit Workflow
Related decisions
ADR-065 · decision context →ADR-066 · decision context →ADR-070 · decision context →ADR-071 · decision context →S4 · E03
Vault and harness configuration gained versioning, recovery checks and clearer provenance. This describes the June custody model; the October operating model is recorded separately.
Historical account from the published journal. Dates describe engineering, not publication.
Context Custody
Related decisions
ADR-061 · decision context →S4 · E02
Live probes and benchmark reruns checked the audit fixes. Missing raw benchmark outputs and a misleading judge comparison exposed why a result needs replayable evidence.
Historical account from the published journal. Dates describe engineering, not publication.
Related decisions
ADR-065 · decision context →ADR-070 · decision context →ADR-071 · decision context →S4 · E04
Operational noise, ingestion work and metadata repairs made a shared review surface necessary. The first Console combined a review tracker, canvas and visible provenance.
Historical account from the published journal. Dates describe engineering, not publication.
Rootweaver Console
Related decisions
S4 · E06
Separate configuration, valid hook responses and visible usage were the first steps. This milestone covers the opening work only; the published episode continues through 18 June, including the later handoff design.
Backfilled through 14 June only. The linked episode also contains later work.
Codex Lane
Related decisions
ADR-074 · decision context →S4 · E07
Codex’s guard read server-provided rate-limit windows, while the Claude guard still estimated usage from local transcripts. The later recalibration and deduplication work is beyond this 14 June backfill.
Backfilled through 14 June only. The linked episode also contains later work.
Usage Guard
Related decisions
ADR-074 · decision context →S4 · E05
Live vault views, voice capture, interactive sessions and explicit approvals joined into an idea-to-session-to-review loop. Terminal access in this period was read-only.
Historical account from the published journal. Dates describe engineering, not publication.
Interactive Runner
Related decisions
ADR-072 · decision context →ADR-073 · decision context →S4 · E09
Several agent-assisted jobs overwhelmed the MacBook. The incident led to a resource guard and a clearer warning when another job would exceed the machine’s capacity. This is the June engineering event told in Season 4, Episode 9; the episode shipped in October.
Historical account from the published journal. Dates describe engineering, not publication.
Resource Guard
Related decisions
Architecture checkpoint
A dated check of code, declared configuration and running workloads: source-graph queries now use PostgreSQL; retrieval uses configured external embeddings; the cluster spans four nodes. Native coding tools, scoped memory and research tools sit around those paths. This checkpoint records what was observed—it is not the date every component shipped.
Architecture checked 14 October · selected paths, not a full health check
The main journal has reached 14 June. This later architecture checkpoint is separate from that historical backfill; some published episodes also follow overlapping arcs beyond June 14.
PostgreSQL Source GraphExternal EmbeddingsLiteLLMVikunjaJupyterLabMLflowLangfuse
Related decisions
Source graph · decision and later context →GPU scheduling · what changed →Cluster growth · decision and later context →