Episode 01 · 28 Oct 2026

Does the loop work?

Iteration 1: a model that never learned, a model that did, and a cloud base that aced fonts and failed on a hand.

TL;DR

Win and failure

The question Can synthetic ink from fonts, plus 25 of one person's lines, teach a recogniser at all: first on a laptop, then on a cloud GPU?

We taught a handwriting reader with fake handwriting drawn from fonts, then showed it 25 real lines from one person. It learned the fonts almost perfectly and still got every second letter of the real hand wrong. Then 10 minutes of training on those 25 real lines halved the error.

On a laptop, the bigger models never learned anything in 20 minutes. A smaller one did.
On a cloud GPU, the first real model got fewer than 1 letter in 1,000 wrong on font ink, and about 1 in 2 wrong on real handwriting.
The problem was the data, not the model: fonts don't write like a hand.
Training a bit more on 25 of the writer's lines halved the error, for 10 minutes on a laptop.

The lesson

A near-perfect score on synthetic data says little about real data. Measure the gap to real data first: a few dozen real examples go a long way.

Why read on Why a deeper model did worse than a smaller one, and the chart where a near-perfect curve turns out to have learned the wrong thing.

1 The question

Can fake handwriting, plus a handful of real lines, teach a model to read one person's hand?

Real handwriting is slow to collect: someone has to write every line and type out what it says. Fonts are free. So the plan was to draw text in handwriting fonts and treat it as pen strokes: synthetic data Training data written by code instead of collected from people: here, handwriting fonts turned into pen paths with timing. Changing it: It's unlimited and licence-clean, but if it doesn't look like real writing the model learns the wrong thing: the synthetic-to-real gap. Here: 300,000 lines for v2, from 134 fonts under open licences (OFL or Apache)., as much as we like.

Train a small network on that, then fine-tuning Continuing to train an already-trained model for a short while on new data: here, your own lines. Changing it: It adapts a general model to your hand from very little data. Too long, or with too high a learning rate, and it memorises your lines or forgets general writing. Here: 40 varied copies of each of your 25 lines, plus 2,000 synthetic lines replayed. It took v2 from 43.6% to 13.1% errors. it (keep training for a short while) on a few lines of the real writer. Nothing had been trained yet, so iteration 1 asked the simplest question: does this loop work at all? First on a laptop for free, then on a rented GPU.

2 Failure: two models that never learned

On a laptop, deeper wasn't better: the deeper models ran out of time before learning anything.

The first two tries stacked 3 layers of 64 cells, about 257,458 weights, trained on 9,667 synthetic lines plus 25 real ones. The laptop CPU got 20 minutes.

In that time they went through the data 4 times (4 epoch One full pass over every training line. Changing it: More epochs keep lowering the training error until it levels off. Past that point they cost time for nothing, or the model starts memorising the training lines. The best epoch is the one that scores best on held-out lines, not the last one. Here: v1's base ran 29 epochs of about 97,000 lines (about 1.5 minutes each on a T4); v2's ran 12 epochs of 290,000 lines (about 12 minutes each). Personalising stops after 6 epochs or 10 minutes.), and their error never dropped below 98%: they read almost nothing. A single, wider layer of 128 cells, with fewer weights (182,962), did learn in the same 20 minutes.

Each point is one epoch. A CER near 100% is what a model scores when it reads almost nothing: the usual first phase of CTC A way to train and read a model that gives one guess per point, when nobody has marked which points belong to which letter. Changing it: It lets the model train on whole lines with only their text. Models trained this way say "blank" almost everywhere, with a short spike for each letter. Here: Merge repeats, then drop blanks: "hh_e_ll_l_oo" reads "hello". training, where outputting only blank CTC's extra output, meaning "nothing new here". Changing it: It separates real double letters (l_l is "ll", ll is "l"). Early in training, models can get stuck predicting only blanks: the blank plateau. Here: One of the model's 178 outputs, next to its 177 characters. is the easy way to lower the loss. The 3 × 64 runs ran out of time before leaving it.

3 The first cloud base: v1

A rented GPU trained the first real model in under an hour, for less than a coffee.

Synthetic lines: 96,667; 51 fonts, OFL and Apache licensed
Model: 3 × 128; 973,490 weights · 1.0 MB as int8
Training: 45 min; 29 epochs on a T4 GPU
Cost: $0.45; per-second GPU billing

This time every setting and the random seed The starting number for every random choice: which font, which word, how much slant. Changing it: Same recipe and same seed give the same data, so a version can be rebuilt. Different seeds show how much a result is luck. Here: Pinned in each recipe. went into a recipe A small file that pins everything that decides what a base model learns: fonts, words, mix, sizes, seed, training settings. Changing it: It makes a version rebuildable and comparable; it's never edited once the version is built. Here: One per base model (v1, v2, v3), each with its seed, so any of them can be rebuilt., a file that rebuilds the run exactly. The synthetic lines were drawn in parallel on rented machines. The run hit its 45-minute limit at epoch 29 of 30.

4 The gap between fonts and a hand

Near-perfect on fonts, wrong on every second letter of real ink: the model learned the wrong thing.

On synthetic lines: 0.09%; held-out font ink
On real handwriting: 51.7%; 95%: 46.1%–57.3%

This is the synthetic-to-real gap The difference between how well a model reads the synthetic data it trained on and how well it reads real writing. Changing it: A model can score almost perfectly on synthetic lines and still read real ink badly, so only a test on real writing counts. Closing it means making the synthetic data look more real. Here: v1's base: 0.1% errors on synthetic lines, 58.3% on yours.. The model had learned to read font skeletons: they lift the pen at every junction, and they never wobble. A real hand does neither.

Nothing about the network could fix that; only the data could. That sentence sets the direction of every episode that follows.

Ochre is synthetic ink throughout the series. The curve says the network can learn; the 52% on real lines says it learned the wrong thing.

5 Win: 25 lines halve the error

Ten minutes of extra training on 25 of the writer's lines cut the error in half.

fine-tuning Continuing to train an already-trained model for a short while on new data: here, your own lines. Changing it: It adapts a general model to your hand from very little data. Too long, or with too high a learning rate, and it memorises your lines or forgets general writing. Here: 40 varied copies of each of your 25 lines, plus 2,000 synthetic lines replayed. It took v2 from 43.6% to 13.1% errors. means training v1 a little more, on the writer's own 25 lines. Each line was copied 40 times with small variations, and 2,000 synthetic lines were mixed back in (replayed Mixing some of the original training data back in while fine-tuning on new data. Changing it: It keeps the general skill while the model learns the new data; too much and the new data has less effect. Here: 2,000 synthetic lines alongside your lines.) so the model doesn't forget the rest. It took 10 minutes on the laptop.

v1 base and v1 personalised on the 24 test lines, with 95% bootstrap intervals
Row	Value	Interval
v1 base	52%	46% to 57%
v1 personalised	25%	20% to 30%

Paired over the same lines: −27.2 points (−31.3 to −22.3), better on 21 of 24 lines and worse on 2. Scored on iteration 3's test set, so every episode compares on the same 24 lines.

6 What iteration 1 shows

The loop works, the data is the bottleneck, and we made three mistakes worth naming.

Shown

The loop works end to end: fonts → ink → network → personalised model.
The data, not the network, limits the base: 0.09% against 51.7%.
Most of what the model knows about this hand comes from 25 of its lines.

Done badly

No prediction was written down before any run.
The laptop model's settings were never written down, so it can't be rebuilt exactly.
The first test set was 15 lines, all English: too few, too narrow.

Why the old scores differ

Iteration 1 was first scored on 15 English lines: v1 base read 58.3% there. Every model was later rescored on iteration 3's 24 lines in four kinds of writing, which is what this series shows.

7 What carries over

Four habits for any project that trains on synthetic data.

Beyond this project

Hold out real data from day one, even a few dozen examples. A validation set made of synthetic data will flatter the model.
If a model sits near 100% error early on, compare its size with your compute budget: a smaller, wider model can learn where a deeper one runs out of time.
Fine-tuning a model trained on synthetic data with a small real set is cheap, and here it halved the error.
Write every setting and random seed into a file from the first run. Settings typed on a command line can't be rebuilt.