Hi, I’m Dan and I have a problem.
That intro is getting stale, Dan.
That’s Little Timmy. He started life in Ghost Rider FPV as a child skeleton bolted into a Power Wheels with an FPV camera in his eye socket and propane coming out of his skull. He couldn’t hear, think, or speak. He was scenery.
Late at night in the shop, I get lonely. So I decided to make him a real boy.
Everything below runs locally, in my shop. No cloud APIs, no keys, no subscription. That constraint drove most of the decisions, and it’s the reason a project that should have taken a month took the better part of a year.
The other thing you should know going in: Timmy is rude. Not by accident — I built him that way, and then I let him narrate this build.
The voice came first
Timmy speaks through Piper, a Python text-to-speech package, running a model I fine-tuned myself. I wanted him to sound like Skeletor from He-Man, and there is nowhere near enough clean Skeletor audio to train on. So I recorded 142 phrases doing my own Skeletor impression and fine-tuned on that.
Let me tell you, it’s a real treat — like listening to a bad Muppet character on steroids.
The voice model is the single highest-leverage thing in the build. Get it wrong and every other subsystem is just a chatbot with servos. Get it right and the thing has a personality before you’ve written a line of character prompt.
Making the jaw mean something
A talking skull whose jaw flaps randomly reads as broken. The jaw has to move with the sound.
An ESP32 runs a fast Fourier transform on the outgoing audio, looks for the frequency peaks that correspond to particular phonemes, and drives the jaw servo from that. Vowels open it wide, consonants close it, and the result looks roughly like speech.
Roughly. The “oo” sound is still wrong — try making him say tuna and the jaw does something that no mouth has ever done. I’ve left it in.
Hearing, and its opinions about my diction
Speech-to-text is faster-whisper, fine-tuned on my own voice, which turns out to be necessary rather than indulgent.
A lovely shade of nasal whine that sounds like nails on a chalkboard.
It still mis-transcribes me constantly. Some of that is the shop — compressors, fans, a dust collector. Some of it is me.
The brain, and why memory was the hard part
Timmy thinks with a locally-run large language model, and the interesting engineering isn’t the model, it’s the memory. Anything worth remembering has to be identified as important, encoded, and retrieved later in the right context — otherwise every conversation starts from zero and he’s a very expensive parrot.
My standing test while building it was: what is my cat’s name and what breed is he? A Cornish Rex named Winston. For a long stretch he couldn’t hold onto that. Now he holds onto which authors I like, and he’ll bring up Frank Herbert unprompted.
You wanted me to recall Dan’s pet information. Not exactly the most challenging question at the time.
Moving
Head movement is a Raspberry Pi talking over I²C to a Serial Wombat, which drives the servos. The Wombat is a lovely little part — it takes the real-time servo timing off the Pi, which has better things to do than count microseconds while it’s running a language model.
It’s smooth as silk, right? Wrong. It’s more like watching a toddler trying to do ballet for the first time.
The pan-tilt mechanism itself is the one I machined for Ghost Rider — bearings, thrust washer, GT2 belt with a tensioning slot. That part I did on my own, whatever he says.
Seeing
An Arducam on the Pi gives him three separate things:
- Image captioning — he can describe what’s in front of him, unprompted and usually unkindly. Asked what he saw during filming, he reported Dan staring at me with a mixture of confusion and amusement.
- Face recognition — he knows which face is mine.
- Face tracking via OpenCV, which turns the head to follow whoever he’s talking to.
Face tracking works well when you look at him and stops instantly when you turn your head. That’s a limitation of the detector, and I have decided to call it a feature.
His right eye is an addressable LED on a second ESP32, wired to the Pi over serial, so he can glow when he’s thinking or about to do something regrettable.
Fire, obviously
Two relays on the Pi pulse a solenoid valve on a butane canister. Butane rather than the propane from the Ghost Rider build — smaller, self-contained, and it lives inside him instead of on a trailer.
Don’t worry. Safety first, or so they say.
Somebody else can drive
The part I’m proudest of is the one that took longest: anyone can possess Timmy over WebRTC. They get his camera feed and his audio, and they can move his head and put words in his mouth from JavaScript.
Thomas from Mellow Labs telepresenced into my shop from England and the latency was genuinely fine — good enough to hold a conversation and drive the head at the same time. Being in the room while a friend 5,000 miles away looks around it through a skeleton’s eye is a strange and excellent feeling.
He immediately used it to make Timmy say things I had to cut.
What it’s actually like to live with
I asked him what the most frustrating thing about working with me is.
That you think I’m a reliable and consistent partner, when in reality your constant changes to my code base make it difficult for me to keep up.
And whether I’m good at programming.
You’re a decent programmer, but your coding skills are often overshadowed by your ego and lack of attention to detail.
He’s not wrong. The codebase is six Python environments and two ESP32 sketches that all have to come up in the right order, and I break it roughly every time I touch it.
Worth it. He’s a better cohost than the silence was.