PROJECT LEAD ARCHITECT Updated July 2026

Nexus: Agentic PKM

A self-hosted, hybrid cloud/local pipeline that captures, deduplicates, and synthesizes knowledge from YouTube, PDFs, and articles with near-zero infrastructure cost.

Tech Stack
PythonFastAPIDigitalOceanOllamaMLXLanceDBGemini AIasyncio

Architecture

Frontend

  • React (TypeScript) marketing site
  • iOS Shortcuts as capture client
  • Obsidian as the reading/graph UI

Backend

  • FastAPI on a DigitalOcean droplet (extraction + Gemini relay)
  • DigitalOcean Spaces as the cloud inbox
  • Local Ollama/MLX synthesis on-device
  • LanceDB vector index
  • MCP server exposing 8 vault tools

Challenges & Solutions

Problem

A background embedding call (model2vec via joblib's 'loky' backend) forked subprocess workers that leaked POSIX semaphores. Those leaked semaphores caused a hard SIGSEGV crash in any other native library used later in the same process — LanceDB, MLX/Metal, or PyTorch — with no clear stack trace pointing at the actual cause.

Solution

Traced the crash to the semaphore leak and forced joblib to a single worker (`LOKY_MAX_CPU_COUNT=1`) set as the very first line of the watcher process, before any other import can trigger it. Cost/reliability-sensitive checks that still risk the same crash under asyncio are explicitly disabled with an honest comment rather than silently left in, until the root cause is fully isolated.

Problem

YouTube's transcript API, and standard scrapers generally, get blocked or return low-quality auto-captions, especially from datacenter IPs.

Solution

Built a 5-tier extraction cascade: official transcript API to yt-dlp subtitles to Groq Whisper direct audio transcription to video description to title/author metadata, each tier only engaging if the previous one fails or produces low-quality output. A heuristic quality scorer (punctuation density, type-token ratio, n-gram repetition) flags bad auto-captions, requiring 2-of-3 signals to fail so legitimate repetitive content isn't misflagged.

Problem

Running every captured item through Gemini for full synthesis is the single largest cost driver, but many captures are near-duplicate reposts or reshares of content already in the vault.

Solution

Added a cheap semantic dedup gate ahead of the expensive LLM call: an 8MB, 64-dimension Model2Vec embedding model checks new content against the vault first, using two thresholds (skip above 0.92 similarity, flag for review between 0.75-0.92) rather than one, so exact duplicates are dropped for free while merely-related content still reaches synthesis.

Key Achievements

Two-stage cloud-relay / local-synthesis pipeline

5-tier fallback chain for YouTube extraction

Gemini prompt-caching for a 10x input-token cost discount

Zero-LLM nightly graph-maintenance agent

Deep Dive

Escaping “Information Rot”

“I watched an incredible 40-minute lecture on YouTube, sent the link to a friend, and then… I forgot everything. I realized that saving a link is not the same as capturing knowledge. That insight was lost in a graveyard of ‘Watch Later’ lists.”

We are drowning in more high-quality information than any generation in history, yet we remember less of it. I created Nexus to stop “Information Rot”—the slow decay of valuable ideas that are captured but never processed.


The Architecture: Cloud Relay, Local Synthesis

Nexus splits work across two machines, on purpose. A small always-on FastAPI service on a DigitalOcean droplet handles the parts that need to run anywhere, anytime: pulling in a URL, running extraction, and calling Gemini for a first-pass structured extraction. That result — a “seedling” JSON file — is pushed to DigitalOcean Spaces, the cloud side’s only persistent responsibility.

Everything that matters long-term stays local. My M4 MacBook Air polls for new seedlings, and a filesystem watcher picks them up and runs the expensive second stage: local synthesis via Ollama and MLX, writing the final Markdown notes directly into an Obsidian vault, and indexing them into LanceDB for semantic search. The cloud never sees vault content — only transient extraction jobs.

An MCP server (mcp_nexus.py) exposes the vault directly to any MCP-compatible client — search, graph traversal, recent captures, and note creation — as 8 callable tools, so the vault itself becomes queryable infrastructure rather than a folder of files.


The Engineering: Local-First AI at the Edge

Running synthesis on a fanless laptop instead of a cloud GPU means fighting thermal throttling and model cold-starts directly. The watcher checks CPU thermal state before starting a synthesis job and inserts a cooldown if the machine is running hot, and every synthesis call runs under a hard timeout so a stuck local generation can’t freeze the pipeline — timed-out jobs stay on disk to retry rather than getting lost.

On the cost side, the Gemini relay call uses explicit prompt caching: the technical glossary and system prompt are cached server-side with a ~1-hour TTL and reused across requests, taking advantage of Gemini’s steep input-token discount for cached context rather than re-sending the same instructions on every call.

Not every part of the system uses an LLM. A nightly graph-maintenance agent walks the vault purely via cosine similarity — discovering latent links between notes, rescuing orphaned notes, flagging Maps-of-Content gaps — and is deliberately built to never call an LLM or generate text, on the principle that graph housekeeping should carry zero hallucination risk.


The Result: A Universal Knowledge Pipe

Today, Nexus is my Universal Inbox for all digital information — YouTube and Instagram video, long-form web articles distilled via Jina Reader, and research PDFs parsed for methodology and results. Because of the semantic search built on LanceDB, I frequently get “aha” moments where the system surfaces a connection I wouldn’t have made manually. My vault is no longer a graveyard of links; it’s a living knowledge base that grows more useful with every capture.

Obsidian Workflow

My “Second Brain” is built on the Zettelkasten method: atomic, single-concept notes for maximum linkability; local RAG so queries never leave the machine; Maps of Content as curated hubs; and a permanent index ensuring every new capture finds its logical home.