Back to Projects
Back to Projects
ON-DEVICE MEETING INTELLIGENCE

OmniMind

A local-first meeting studio for iOS — live transcription, semantic search over everything you've ever captured, and AI summaries, with audio, transcripts, and vectors that never leave the device.

github.com/Thet9354/OmniMind
OmniMind — on-device transcription, semantic search & AI summaries

The Thesis & The Problem

Meetings generate the most important context in a working week and lose almost all of it. The usual fix is a cloud recorder that ships your raw audio to someone else's servers for transcription and summarisation — a privacy trade most people make without thinking about it. OmniMind refuses that trade. Everything — audio, transcripts, embeddings, and generated text — stays on the device by architecture, not by policy.

It runs live transcription as you speak, builds a semantic memory of every meeting you've ever captured, and answers questions across all of it. “When did we decide the pricing?” returns the exact moment, ranked by meaning rather than keyword match — computed entirely on-device with Apple's on-device speech, embedding, and language models.

Local-first, provably: the capture, transcription, semantic-memory, and synthesis subsystems are each isolated behind a Swift 6 strict-concurrency boundary. Every AI failure path lands on a working fallback — no model degrades to extractive summaries, missing assets still persist the capture, a refused generation leaves the raw transcript standing.

Architecture: Four Isolated Subsystems

The pipeline is a strict data-race-free graph. All cross-boundary traffic is Sendable DTOs — never live models, never audio buffers, never attributed strings. The audio render thread does exactly one thing: a lock-free enqueue. Backpressure sheds the oldest audio rather than blocking real-time capture, and every drop is counted and surfaced to the user as a degraded-capture warning.

AVAudioEngine (render thread) │ lock-free, bounded, drop-oldest ▼ AudioBufferBridge ─→ TranscriptionActor (SpeechAnalyzer, on-device) │ TranscriptSegment (Sendable) ▼ SegmentCoalescer ─→ EmbeddingStore (@ModelActor) │ NLContextualEmbedding → vDSP cosine scan ▼ MeetingSynthesizer (FoundationModels) summaries · Q&A · transcript repair

Engineering Invariants

  • Bounded live memory by construction: a 3-hour meeting costs the live view the same memory as a 3-minute one — long transcripts render through windowed fetches.
  • Atomic vector writes: all persistence flows through one @ModelActor; embeddings commit atomically with their rows, and a backfill pass repairs any row whose ML assets were missing at capture time.
  • Verified monetisation: entitlements derive only from cryptographically verified StoreKit transactions — the subsystem is fully built and tested, dormant behind a pilot flag while feedback is gathered.

Testing

58 tests across 15 suites (Swift Testing): sample-rate-conversion convergence, bounded-bridge shed accounting, a TTS-synthesised golden-audio pipeline gate (word error rate ≤ 0.15) that needs zero hardware, a 50,000-vector semantic scan under 50 ms on the Accelerate hot path, a six-stage StoreKit lifecycle over one test session, and structural memory-bound proofs on a synthetic 3-hour session.