TrueFlow

Overview

TrueFlow is a system-wide voice-writing tool for macOS. You hold Option+Space in any app and speak; on release, the speech is transcribed by Deepgram, normalised to the active project's vocabulary, reshaped by a chosen Voice and Mode through OpenAI, and inserted at the cursor. A second shortcut rewrites a selected passage from a spoken instruction.

The code is complete up to Milestone 7 of its plan, but it has not yet been validated on physical Macs or against real provider and account services, and billing and release infrastructure are not built.

Context and problem

The MVP plan aims first at authors, fiction writers and long-form writers who care about a consistent writing voice, while staying useful for ordinary dictation anywhere on the Mac.

Its premise is that basic speech recognition is becoming a commodity, so the value has to come from what follows it, from intent and project vocabulary to a writing Voice and a Mode, with Voice and Mode kept independent.

The hard engineering sits at the edges: getting audio from the microphone to a streaming provider, and then putting text into someone else's app at the cursor without losing or corrupting anything.

My role

I am the only developer. Both commits in the repository are mine: the initial commit and a pull request that added microphone selection.

That covers the Rust core, the macOS platform layer, the provider integrations, storage, the React UI, the tests and CI, and the design documents.

Constraints and requirements

• Platform: macOS only for the MVP, Apple Silicon first, with no Windows, Linux or mobile builds.
• Secrets stay in Rust: the webview never sees API keys or provider tokens; React only gets state snapshots and scalar meter events.
• Privacy: logs carry metadata only, audio never touches the disk or React, and History stores text only, with no audio, prompts, selections, clipboard contents or credentials.
• Never lose the user's words: a failure has to keep the text or surface it, not drop it quietly.

Research and planning

The work follows an MVP plan that sets the product thesis, the platform scope and nine milestones, from the repository foundation to a paid beta.

A progress log tracks the milestones, a test matrix keeps automated results apart from the manual macOS checks still to run, and 26 numbered architecture decision records explain the choices.

I also built a benchmark harness for speech-to-text providers, because the first provider was chosen on its API features rather than on measurements.

Solution and user experience

Hold Option+Space in any app and a small native overlay appears, also over full-screen apps and across Spaces; release it, and the processed text is inserted where the cursor was.

Each project has its own vocabulary. It is sent to the speech provider as hints, but a deterministic normalisation pass in Rust has the final say.

A project also selects a Voice and a Mode, which the OpenAI step applies; if that step fails, the normalised transcript is inserted anyway with a warning.

To rewrite, the user selects a passage, holds the rewrite shortcut and speaks an instruction. The passage is replaced only if the selection still checks out, and otherwise nothing changes.

The first run walks through permissions, account, shortcuts and one real dictation, and setup is not marked ready until that dictation succeeds. After that the app runs from the menu bar, with Settings and a local History.

Key features

• Hold-to-talk dictation into any macOS text field, with a native overlay across Spaces and full screen.
• Project vocabulary: advisory hints for the speech provider, authoritative normalisation afterwards.
• Voice and Mode transformation that falls back to the normalised text.
• Selection rewrite by voice, which fails closed and revalidates the selection before replacing it.
• Onboarding, Settings, History and a menu-bar mode, with onboarding gated on a real dictation.
• Accounts: a browser/device sign-in flow, with the session in the macOS Keychain and provider tokens only in Rust memory.
• Microphone selection by stable device ID, falling back to the system default.
• Developer tools: a speech-to-text benchmark harness, and an insertion probe that shows which input transports an app accepts.

Architecture and stack

TrueFlow is one modular Tauri 2 app. Rust owns audio, hotkeys, providers, storage and secrets; React gets state snapshots and scalar meter events over the bridge and never sees raw audio or keys.

• Core: Rust with Tauri 2 commands, a Rust-owned global-shortcut lifecycle, a menu-bar tray, tracing logs and Keychain credentials.
• Audio: CPAL capture at the device's native rate, with bounded mono buffering in memory.
• Speech-to-text: a Deepgram Nova-3 streaming client behind a provider-neutral transcription coordinator.
• Transformation: an OpenAI adapter behind a provider-neutral interface.
• macOS layer: AVFoundation microphone authorisation, Accessibility permission, and cursor-safe insertion.
• Storage: SQLite owned by Rust, with five embedded migrations for projects, vocabulary, Voices, Modes, History and the microphone preference.
• UI: React 19, TypeScript and Vite, tested with Vitest and Testing Library.
• Quality: a Rust integration test for the streaming pipeline, and GitHub Actions CI.
• Telemetry: opt-in hooks for a PostHog-compatible analytics endpoint and a crash-intake gateway, set by environment; no backend for them is in the repo.

Implementation details

When the shortcut is pressed, Rust captures the active project, vocabulary, Voice and Mode as one immutable snapshot, so changes made while speaking only affect the next dictation.

A sample-bounded queue sits between the realtime audio callback and the network task. If it overflows, the operation fails with an explicit error instead of dropping speech and reporting success.

The app reports completion as soon as Deepgram delivers the final transcript and closes the WebSocket afterwards, so the user is not kept waiting on the teardown.

Insertion is a chain: direct Accessibility insertion first, then synthetic typing, then one pasteboard write with Command+V, and finally clipboard-only recovery.

The automated suite passed on 2026-10-12: 163 Rust unit tests, 2 streaming integration tests, 26 frontend tests and a production Tauri build. None of that replaces the manual macOS checks, which are still pending.

Challenges

• Editors that swallow writes: VS Code, Sublime Text and Office accept a direct Accessibility write and then discard it. Direct insertion now compares the caret before and after the write and falls back when the write was discarded.
• Held modifier keys: Option was often still down when the paste ran, so Command+V arrived as Command+Option+V. The paste now waits a bounded time for modifiers to clear.
• Clipboard coupling: one dictation's leftover transcript was treated as the user's clipboard by the next. The backend now records the change count of each transcript it writes, so it recognises its own.
• Paste in Microsoft Word: after several corrections, identical logged dispatches still gave different results, so I put synthetic typing ahead of paste.
• TLS set-up: the first real Deepgram connection panicked because no rustls crypto provider was selected; the app now installs one explicitly at start-up.
• Permissions: microphone and Accessibility permissions had to be handled natively in the macOS layer.

Trade-offs

• The design appears to trade polish for safety: if transformation fails, the plain normalised text is inserted, while a failed rewrite changes nothing at all.
• Discarded-write detection needs positive proof of failure. Controls that report no caret are treated as inserted, because a false alarm would paste the text twice.
• Selection rewrite never uses synthetic typing, because typing in chunks would destroy the selection before the replacement could be verified.
• A disconnected microphone falls back to the system default instead of failing, because losing speech is worse than recording on a different input.
• Deepgram is a provisional choice made on its API features; no provider benchmark has been run yet, which is what the harness is for.

Links

• Source Source code github.com

Skills demonstrated

• Native desktop development in Rust with Tauri 2.
• macOS platform integration: Accessibility, AVFoundation, the pasteboard and synthetic input.
• Streaming speech-to-text over WebSockets, with bounded queues and explicit failure states.
• LLM text transformation with a safe fallback.
• Privacy-minded handling of secrets, logs and local data.
• Architecture decision records and milestone planning.