Tell Me A Story
Capturing the stories we make up at bedtime.
A local-only system that records our improvised bedtime stories and turns them into honest, speaker-labeled transcripts — toward the point where every story is captured and you can see the characters, themes, and arcs running through them. Nothing leaves the house.
by Saurin Choksi — Creative Technologist · Emmy-nominated children's animation writer (PBS, Nickelodeon, Netflix)
View on GitHub (opens in new tab)What it does
A story gets told; a recorder catches it. A phone for now, a screenless bedside gadget later. Storytime itself stays tool-free.
Local models work out who said what, word by word. Everything runs on machines in the house; the audio never leaves.
The machine flags names it mis-heard — invented characters especially — and asks. I rule yes or no. Not a word changes until a fixer earns trust.
The destination: every story captured — and the characters, themes, and arcs my daughter invents visible across all of them.
Origin
My daughter shouts "Tell me a story!" 1,000 times a day at me. Then at bedtime, I'm commanded to make up three more before she'll go to sleep. Folks, it's the end of the night. My brain's melted. I love my four-year-old but telling "one more story" feels like a type of tiny torture.
Still, when it's done, I'm always glad we share this ritual. And sometimes, I'm impressed by what we create together.
Pappy, the boy who looks like whatever food he wants to eat, usually a quesadilla. Too good! A perfectly executed Hero's Journey arc about a talking fork. Who knew it could be done?
Those bedtime stories are told and gone. I decided I wanted to keep them.
Why and for what, I'm not sure yet. One side of my brain complains, "Do we really have to digitize everything?" But a little voice insists, "Build the thing. Do it your way. Keep these stories. See what comes next when we get there." Ok...
How it works
Three parts: how a story gets in, what happens to it, and the screens where I check the machine's work. The pipeline below is the deep cut — click any stage.
Right now, capture is voice memos on a phone. It works, but a phone doesn't belong in that calm, quiet bedtime space.
When I lay my head down on my daughter's silly Elmo chair, I want to keep it dark and calm. The plan is an ESP32 device (a tiny, cheap microcontroller common in smart-home gadgets) — screenless, dark-operable, tap once and it just works.
Local ML models turn recordings into speaker-labeled transcripts.
Everything runs on-device for family privacy.
Currently running on Apple Silicon. Next phase is a Jetson Orin Nano (a small AI computer that can run models continuously without a laptop in the loop).
There's a Session Reviewer tool for validating transcripts against audio, and a Monitor that flags mis-heard names across every session for review.
For the real UI... I want to be able to SEE the stories. What that means exactly, I'll discover over time.
The Pipeline
Messy audio in, structured transcript out. Local processing.
A phone recording today; a screenless bedside device later. The pipeline scans for new audio and sets up a session.
The test recordings are of me telling Arti stories from the Mahabharata, recalling the ancient Indian epic I loved as a kid. It's crammed with Sanskrit names that make speech recognition systems sweat. Let's see what the models make of the Pandavas and Kauravas.
Whisper runs locally on Apple Silicon. It produces segments with word-level timestamps, which is what makes speaker alignment possible later. It handles Dad fine. But it doesn't know Sanskrit, and a four-year-old's pronunciation doesn't help.
Two separate repairs happen here first.
Toss the made-up words. Whisper fills quiet gaps with words nobody said, stamped over silence with nothing really under them. Out they go.
Whisper learned from captioned video, so silence makes it pad with lines like this.
Re-time the rest. Whisper marks the first word of a segment too early, parked in the pause before it's actually spoken. Lining every surviving word up against the audio slides each one onto the moment it's really said.
The start marker slides out of the silence, onto the word.
Pyannote listens to the audio and maps out who is speaking when.
No words — just stretches of audio labeled by voice
Each word's timestamp lands it inside a speaker block — that's how it gets labeled. Pyannote doesn't know names, just SPEAKER_00 and SPEAKER_01. Real names come later, once the pipeline matches voiceprints to known speakers.
Sometimes pyannote heard a voice
but Whisper couldn't decode the words, especially when Arti is getting sleepy and her
voice drops to a murmur. The pipeline marks these moments
[unintelligible] rather than pretending nobody spoke.
This stage notices the names Whisper mangles and, instead of guessing a fix from the spelling, listens again. It works out which story world we're in from the names alone, asks a small local model for that world's cast, including group names like "Pandavas", and replays a few seconds of audio around each suspect name with that cast in Whisper's ear. Three fixers work this way: one for storybook casts, one for our own family names, one for an invented name spelled several ways in one night.
None of them writes anything. The whole stage runs in a watching mode: every match becomes a question on a review screen, labeled with why the machine believed it, and I rule on each one — yes, "Bushma" means Bhishma; no, "arrows" is just the word arrows. Even a yes doesn't change a word; one deliberate apply pass comes later, after whole groups of questions have earned trust. The first night of rulings showed why: the group that looked strongest held the one dangerous guess — "Jammus", an engine Arti invented, offered up as the real Thomas & Friends engine "James". They sound nearly identical. Only one of them is hers. The fixer asks, I decide, and nothing writes until trust is earned.
The pipeline builds a voiceprint — a compact fingerprint of how each speaker sounds. It compares that fingerprint against known profiles, including ones built from earlier recordings, so a returning voice is recognized from a single sample. High-confidence matches are assigned automatically; the system learns Dad's voice from Arti's without being told every time.
When the pipeline encounters a voice it hasn't heard before, or isn't confident about, a dedicated speaker review page lets a human step in — confirm a match, create a new profile, or reassign. Those decisions persist across sessions and sharpen the voiceprint, so the system gets more confident over time.
Named speakers, accurate words, honest gaps. This is the file everything in Beyond the pipeline is built on.
The pipeline produces a transcript. But how do I know any of it is right? I built a tool to find out — play back audio against the transcript word-by-word, catch hallucinations, leave notes on anything suspicious.
Click to enlarge
After transcription, the pipeline goes back over its own output looking for mistakes — mis-heard names most of all, plus a few failure modes the models reliably have. Nothing is changed automatically; each check only raises a flag for me to review. Building them taught me where plain code is enough, and where a model earns its place.
One bedtime recording often holds several stories, and the name checks work best one story at a time. So the pipeline splits first — using cues like a long pause or a line such as “once upon a time” or “the end,” then a small local model (Qwen 3.5 4B) confirms the real breaks. For each story it then makes a call: a world it knows (Thomas & Friends, the Mahabharata), something wholly made up, or a known world with characters invented on the spot. Knowing the world is what lets the next step tell a misspelled real character from one we invented.
Whisper mangles the family’s own names constantly — my daughter’s name alone gets heard a dozen ways. This detector checks each name against a known family roster, no model required. On recordings it had never seen, it caught every mis-transcribed name a careful re-listen turned up, with a single false alarm.
How plain code beat the AI models →A character we invent — say a train engine named “Jammus” — gets spelled a different way each time it’s said, so it’s hard to follow through the telling. Plain text-matching catches most. The hard ones are invented names that are also ordinary words (my daughter has a character called “Bibi”), so those go to a small local model that reads a few lines and decides whether the word is being used as a name.
Where a model earns its place →Inside a world the model knows, it lists who belongs there and flags the transcriber’s misspellings. In a Mahabharata story it knows the cast includes Bhishma, so when the transcript says “Bishma” it catches it.
Whisper sometimes invents words during silence. Two independent systems disagree — that's the signal.
Diarization: no speaker detected, coverage 0.0
→ Flagged as hallucination
"Well." — probability 0.133, fabricated
→ Two consecutive identical words. One real, one not.
Diarization detects Arti's voice at three points where Whisper produces nothing. The pipeline keeps those spots rather than dropping them.
Whisper: [no words produced]
→ Marked [unintelligible]
Whisper's tiny model produces absolute silence where Arti speaks. The large model recovers her voice. With the tiny model, ten seconds of her talking simply vanish.
large: "Dad, why do the fondos and the goros want to be king?"
Every flag lands on one screen, across all sessions, each with a play button to hear the exact moment. When a detector is wrong — it once flagged our invented “Jammus” as a misspelling of the real “James” — I overrule it, and the correction sticks through every future scan. Like the Session Reviewer, the Monitor only points; it never changes the transcript.
Click to enlarge
Where the pipeline is headed next.
Story Element Extraction
Pull characters, events, and relationships from finished transcripts.
Story Visualization
A way to actually see the stories we've told — the form is still to be discovered.
AI & Kids
Once I have transcribed stories, the generative AI applications seem easy, obvious, and off-putting. Extract recurring themes and characters using a sprinkle of local model intelligence? (Seems fine) Generate Nano Banana illustrations of characters in the style of famed Pixar illustrator, Sanjay Patel? (um.. not cool) Build an Eleven Labs powered penguin companion stuffy that tells stories with a voice that sounds exactly like Daddy? (OH GOD. WHAT HAVE I DONE)
Models were trained on creatives' work without permission or money, but the tech is here. It's not disappearing. Our kids will grow up with it. What should its place be in their lives? How do I approach building on such a fraught foundation? (Meanwhile, Choksi, you use AI for coding every day... what about that IP? Hypocrite!)
What does it do to a kid when their thoughts skip straight to a generated image? Isn't the whole point to have those ideas live in your head, and then if you decide to put in the effort, pick up a crayon and be delighted with what your hands can make? What happens when a four-year-old forms a relationship with something that talks back to her whenever she wants, optimized to build attachment, before her brain is fully cooked?
So where do I go once the audio → transcripts system is built? Not sure. I know the point isn't to outsource storytelling, creativity, or imagination to machines. It's to understand our voices as storytellers. And have fun.
I do believe this project will help me clarify my own red lines about AI/technology and kids. As I work on this, I'm talking to child development experts, artists, and writers in my children's media, animation, writing, and edTech networks to help sharpen my thinking.
Go Deeper
- "AI Is Breaking Education" — Your Undivided Attention with Rebecca Winthrop
- "AI in the Toy Box" — Common Sense Media (2026)
- "The Rise of AI Companions" — EdTech Insiders with Dr. Sonia Tiwari
- "Beyond the Engagement Trap: A New Design Architecture for Children & AI" — Michael Preston, Joan Ganz Cooney Center
Guiding Principles
Immediate Connection
Removing friction brings creative choices you could not have imagined before.
Ideas start out tiny, weak and fragile. — Victor
Hard Fun
Harness the passion of the learner to the hard work needed to master difficult material.
I have no doubt that this kid called the work fun because it was hard rather than in spite of being hard. — Papert
Calm Technology
Technology can live on the periphery, there when you need it, and otherwise invisible.
If computers are everywhere they better stay out of the way. — Weiser
The Arti Test
I only work on things for kids that I'd give my child.
Daaaaad, tell me a story whiiiiile I'm on the potty. — Arti
Learning
Part of why I am excited about this project: I get to learn things I actually want to learn.
- AI-assisted development (Claude Code)
- Audio ML pipelines (Whisper, pyannote)
- IoT capture devices (ESP32)
- Edge ML deployment (Jetson, CUDA)
- Local-first architecture
When I realized this bedtime story project gave me a reason to explore NVIDIA's CUDA/Jetson stack, I was unreasonably happy. Robot-brain tech! Positronic! That's exciting. And I do want to have a voice in the rooms where these things that will interact with us — our families, our kids — get built.