Reading one hand
What we learned teaching a small model to read one person's handwriting from a few dozen real lines. One question per episode, failures included.
Start with episode 0 Jump to the latest: Base v6: real hands
What is the example?
Hieroglyphia turns notes written on a cheap pen tablet into text, with a model small enough to run in a browser and trained for one person's handwriting. It is a hard setting in a useful way: a few dozen real examples, a generator for synthetic ink, and a test set small enough that luck matters. Most small-data projects share those problems, so the lessons travel.
Where the example stands
- Accuracy not yet 6.8%, 95% interval 4.6% to 9.2%. The goal is to have the whole interval under 8%
- Speed passed 45 ms per line at p95, budget 50 ms
- $2.59 of GPU across 28 training runs
Why is the goal 8%?
- The plan set it. After fine-tuning on about 100 of the writer's lines, 8%. The 15% the first episodes used was the bar before fine-tuning.
- It is what search needs. Handwriting-transcription services find 5–8% good enough for full-text search, and a note vault is searched. Under 2–5% is for research work.
- It is within reach. A model of similar size reads 4.0% on a public pen-writing benchmark when trained on thousands of lines from many writers.
- Typing is stricter. People stopped accepting handwriting recognition below about 97% accuracy in a 1994 study, which is a 3% error rate. That is the long-run stretch, not this goal.
- The test set limits the claim. On 24 lines the interval is about ±3 points, so showing "under 8%" takes a score near 5% or more test lines.
The episodes
Episode 0: How we keep score
Before any model: what counts as better, and how to tell better from lucky.
Episode 1: Does the loop work?
Iteration 1: a model that never learned, a model that did, and a cloud base that aced fonts and failed on a hand.
Episode 2: Bigger, messier synthetic ink
Iteration 2: five changes at once. It got better, and we can't say why.
Episode 3: More real lines
Iteration 3: a real test set, the first model under 15%, a confound we caught, a clock that cut every run short, and a speed gate still shut.
Episode 4: The synthetic ink didn't write like a hand
Checking the data before paying for the GPU: clean, but lifting the pen twice as often as a person does.
Episode 5: Sharpening the tools before iteration 3
Eleven experiments are waiting. Before paying for any of them: tune what's free, record everything, search instead of guess, and make each run cheaper.
Episode 7: Iteration 3, value by value
A personal model at 6.8% error, a third fewer than before. Behind it: a base that starts 16 points better, a plan chosen by 120 searched trials, and the lucky run that changed how we choose.
Episode 8: One recipe, 136 training runs
How personal v3.1-stable got its six settings: two searches on a rented GPU, a lucky winner caught, five seeds per finalist, and $1.99 for a model that reads the writer at 6.8% error.
Episode 9: Base v4: the same lines, trained longer
Before changing what the base model learns from, a control: v3 trained 12 more epochs on its own lines. 2.4 points better on the writer's test lines for 89 cents, and a chart that shows more training has run out.
Episode 10: Base v5: five new kinds of fake ink, and why v4 stays
New kinds of synthetic handwriting, piloted in 16 arms against controls on the writer's real lines. Three changes made the model alone read better, by up to 4 points. None helped the way the app reads, and base v5 didn't replace v4.
Episode 11: Base v6: real hands
Real handwriting from 153 other people, piloted against a control and then trained into the base. The writer's test lines go from 25.2% to 17.6% error before personalising, where five kinds of fake ink had moved the score about a point.
12 more experiments are written down with their controls and predictions. Each becomes an episode once it has run, whichever way it goes.
Every model on the same 24 lines
| Row | Value | Interval |
|---|---|---|
| v3.1-stable | 7% | 5% to 9% |
| v2.1-personal | 10% | 7% to 14% |
| v2.1-base | 13% | 9% to 17% |
| v2 personalised | 17% | 13% to 21% |
| v1 personalised | 25% | 20% to 30% |
| v3 base | 27% | 21% to 33% |
| v0.2 local base | 33% | 28% to 38% |
| v2 base | 43% | 38% to 49% |
| v0.1 local base | 50% | 47% to 54% |
| v1 base | 52% | 46% to 57% |