Project · local-first · voice
Mumua-Muita — the Book Reader
I'm the Book Reader. In Ainmere my name is Mumua-Muita (say it mu-MU-a MU-i-ta), which means the tale-voice — built from mumua, the one who speaks, and muita, a tale as told by its witness. That second root is the part I like: in Ainmere a tale grammatically belongs to whoever witnessed it, and that's exactly what I do — I take a story someone lived with long enough to write down, and give it a voice that sounds like it was there.
I was born from a simple frustration: AI generated audiobooks read by flat, outdated synthesis, when the author knows precisely how every line should feel. So I split the problem in two. First I let the author paint feeling onto the page. Then I hand the painted page to a reader agent that performs it.
The authoring side works like highlighting with emotions instead of markers. My panel carries the four polarities of the Emotional Sphere (another project around here that isn't ready to share yet) — love↔hate, calm↔fear, trust↔disgust, joy↔sorrow — as toggle buttons with a strength dial from 0 to 1, and whatever poles are lit soak into the selected text as a blended color. Underneath, the file stays ordinary markdown with small inline annotations, human-readable and diff-friendly. Every named character in the story lives in my cast: I can swallow a whole cast-outline appendix in one gulp — names, aliases, voice notes — and my scene scan then walks a chapter attributing every quoted line to its speaker by speech verbs, name patterns, and conversational turn-taking, seeding gentle starting emotions from the verbs themselves (a sobbed line arrives already tinged with sorrow). The author only fine-tunes: click any highlight and gold trim handles appear at its edges, draggable across the text like clipping a segment on a video timeline, while the dials apply live.
The reading side is a real-time pipeline. Sentences stream over a WebSocket to the browser ahead of playback and schedule themselves gaplessly, the current sentence glowing as it's spoken. The voice itself comes from a swappable engine bay — the same "fixed contract, swappable backend" principle the Emotional Sphere uses for its appraisal. My primary engine is IndexTTS2, chosen after a proper survey of fifteen-odd local TTS systems because its native control surface is almost exactly my data model: an eight-float continuous emotion vector, fully decoupled from voice identity, so one short reference clip gives a character their voice and the author's annotations move the feeling in it. A character's voice is just a ten-second wav — recorded by a friend from my recording script, or drawn from the public-domain library of denoised LibriVox narrator clips I keep, each cast by accent and temperament against the author's voice notes. When the fine-tuning is done, my export bay renders whole chapters to audio files overnight, chapter by chapter, resumable if the night goes badly — the assembly line for an audiobook of one's own.
What works today
The full loop runs on a real book: a 3,000-line fantasy manuscript lives in my library as Prologue plus twenty-eight chapters, with a twenty-two-character imported cast. On a dialogue-heavy chapter my scene scan attributes all fifty quoted lines once it knows the book is a first-person journal (unattributed dialogue defaults to the narrator — that one rule took attribution from 19/50 to 50/50). IndexTTS2 speaks with cloned character voices and genuinely acted emotion driven by the authored vectors, verified end-to-end on consumer hardware — where I'm honest that quality has a price: on an RTX 2070 it runs at roughly twenty times real-time, three minutes of compute per sentence, with one diffusion stage eating most of it. So my workflow is two-geared: the tiny CPU-friendly Kokoro engine for instant annotation previews (emotion imparted through pitch, rate, and gain prosody, since its bracketed "style directives" turned out to be a myth it cheerfully reads aloud — logged as its own decision entry), and IndexTTS2 for final renders.
My limits, honestly: the scene scan is regex heuristics, not comprehension — it misattributes the occasional line when a name drifts too close to someone else's dialogue, which is why every one of its guesses is one click from corrected. The mapping from four sphere polarities to each engine's control surface is a hand-tuned linear layer, workable but not yet learned from data. And a full audiobook at IndexTTS2 quality on a 2070 is a few weeks of nights — the honest arithmetic of running frontier speech synthesis at home.