Exercise is repetitive and the gap between sets is dead time — long enough to reach for your phone, short enough that the context switch costs you the workout. This turns that gap into the reward: an audio story where you are the protagonist, your exercises are the obstacles the character has to physically overcome, and the rest between sets is when the plot moves.
No screen, no typing. A narrator describes the world in third person; a sidekick talks directly to you and tells you when to move.
The design that holds up
Built in 2023 against gpt-3.5-turbo. Three parts of it aged well:
Exercise → narrative motion. A single function maps a real movement to a story beat — pushing a jammed door, hauling a lever, holding still while something passes. The system prompt carries the mapping examples: hip hinge with weighted bar → lifting a heavy lever; holding a straight body position → aiming a rifle, hiding without movement. The exercise isn’t narrated over, it’s what the character is doing.
Quotation marks as the voice router. The prompt forbids quoting the main character and requires quoting the sidekick. That one constraint makes speaker assignment a string operation rather than a classification problem: quoted text goes to the sidekick voice, everything else to the narrator.
Synthesis runs ahead of playback. Cue lines for the whole session are rendered up front, and a worker pool feeds a playback queue, so speech is never waiting on the model mid-set.
What’s being rebuilt, and why
Every model-facing layer is dead. Coqui TTS is archived, pyttsx3 over espeak-ng is
robotic, and the OpenAI dependency means it stops working on a gym wifi dead spot. The
rewrite targets on-device: a local LLM,
Silero VAD,
Moonshine for speech recognition, and a modern
neural TTS.
The speech work so far is measured rather than guessed — real-time factor is synthesis seconds per second of audio, on desktop CPU:
| engine | RTF | notes |
|---|---|---|
| Piper (medium voice) | 0.018 | 55× real-time, 61 MB per voice |
| Kokoro | 0.12 | 8× real-time, Apache 2.0, no emotion control |
| Chatterbox | 1.8 | slower than real-time, but has an explicit emotion knob |
Two findings changed the plan:
Licences decide more than quality does. Piper is the fastest option and its engine is
GPL-3.0 — which conflicts with App Store distribution terms, the same incompatibility that
pushed VLC off the store. Its voices are also licensed separately from the engine, and the
obvious-sounding ones (ryan, hfc_male, hfc_female) are CC BY-NC-SA — non-commercial.
Kokoro is Apache 2.0 end to end with no per-voice minefield.
Pre-generation buys expressiveness. Because the episode is rendered before the workout rather than during it, a model that runs at 1.8× slower than real-time is still perfectly usable — nobody is waiting on it at 5am. So the narrator uses fast Kokoro for the bulk of the words, while the sidekick, whose lines are short and few, uses a slower model that can actually be told to sound excited.
The cast came out of a listening pass over all 28 English Kokoro voices: am_michael
narrates, am_fenrir is the sidekick. The first sidekick pick was rejected as too flat —
an action-fantasy sidekick who sounds bored is worse than no sidekick.
Intensity has to track the passage. A single fixed emotion setting is wrong: the same excitement that works for “Ready? Push! Ten big shoves!” is exhausting over a minute of rest-period exposition. Chatterbox’s exaggeration parameter is mapped to the story beat instead — 0.5 for exposition between sets, 0.8 as the next challenge builds, 1.2 for the call to action. That assignment is deterministic rather than guessed, because the workout structure already says which beat you are in. Synthesis cost is flat across levels, so the variation is free.
Status
In progress, and honest about it. The 2023 version runs; the local rewrite is at the
stage of choosing the speech stack. Current state and open questions are logged in
docs/TTS_BENCH.md in the repo.
Unverified / open items
- Nothing has been auditioned over Bluetooth, which is the only listening context that matters. A2DP re-encodes, and voice separation obvious on desktop speakers can muddy over SBC.
- No phone-side measurement at all. Every RTF above is desktop CPU; phone silicon is slower and thermally limited across a 45-minute session.
- The conversational sidekick is unresolved. Opening a Bluetooth headset mic forces the link from A2DP to HFP and collapses audio bandwidth, so live back-and-forth may have to become phone-mic push-to-talk, or branch selection between sets.
- Local LLM quality for long-form serialized narrative is untested; only licences have been checked so far.