A Holiday Accident That Quietly Broke Machine Learning's Oldest Rule

The most interesting discoveries in science are not the ones you planned. They are the ones you almost deleted.

Late 2021. A research team at OpenAI finishes a training run, sees the usual bad news—model overfit, generalisation terrible—and leaves for the holidays. The machines keep running.

Days later, they check the logs. Something is wrong with the graph. Or rather, something is impossibly right.

The model that had failed completely on unseen data now got everything right. Every single test case. Not gradually. Abruptly. Like a switch had flipped somewhere inside the weights.

They called it grokking. And it quietly broke a rule that machine learning had followed for decades.

Start Here If You Are New to Machine Learning

Think about how you studied for the hardest exam of your life. You memorised every past question, every formula, every worked example from the textbook. Word for word. You walked into the exam and aced it.

Then your professor gave you a slightly different version of the same problems. New numbers, same underlying logic. You blanked. Not because you were not smart—but because you had memorised, not understood.

Neural networks do exactly this. When you train them on examples, they will often choose the easiest path: memorise the answers rather than learn the rules behind the answers. They become brilliant parrots on familiar data and hopeless strangers on anything new.

In machine learning, we call this overfitting. Training accuracy goes to 100 per cent. Test accuracy stays near random chance. Every textbook, every course, every experienced engineer will tell you the same thing when this happens: stop training. It is over. The model has overfit.

That rule held for decades. The 2022 grokking paper did not bend it—it snapped it in half.

The Experiment That Broke the Rule

The OpenAI team was working on something deceptively simple: training small neural networks to do modular arithmetic. The task was problems like this—what is (17 + 38) mod 113?

Modular arithmetic is like clock arithmetic. On a 12-hour clock, 10 + 5 does not equal 15. It equals 3, because you wrap around. The mod 113 version has 113 possible answers. Random guessing gets you roughly 1 per cent accuracy.

The models overfit quickly and predictably. Training accuracy hit 100 per cent. Validation accuracy—how well the model handled new, unseen equations it had never been shown—sat around 5 per cent. Not much better than guessing.

The researchers went on holiday. The models kept running for days. Hundreds of thousands of extra gradient steps past the point where anyone would have said enough.

When they came back and looked at the logs, validation accuracy had jumped. Not crept up. Jumped—from near-random to near-perfect, almost overnight on the graph.

The Graph That Changed the Conversation

This is what the training history actually looked like. Read it slowly—each phase tells a different part of the story.

What Each Phase Is Actually Telling You

The graph has four distinct chapters. Most people only ever see the first two.

The plateau is the part that makes this genuinely unsettling. It can last for thousands of steps. From the outside, nothing appears to be happening. The model looks completely dead. The curves are flat. Any dashboard or monitoring tool would tell you to cut the run.

And then, without warning, the orange line rises like a cliff.

Why Does This Happen?

The honest answer is that researchers are still building the full theory. But the leading explanation is elegant enough to be worth understanding.

When a model memorises, it does so using large, complicated weights—essentially hardcoding specific training answers into its parameters. Think of it like writing every exam question and answer on a huge, messy cheat sheet. It works perfectly for the questions you memorised, and for nothing else.

But gradient descent has a quiet preference for simplicity. It creates constant, gentle pressure to shrink the weight values over time. Large, complex weights cost more. Small, efficient weights cost less.

Here is the key insight: the generalisation solution—the one that has actually understood the mathematical rule—can be expressed with much smaller, cleaner weights than the memorisation solution. It is like the difference between a cheat sheet with ten thousand individual answers and a single clean formula that generates all of them.

Given enough time, the pressure to reduce weight magnitude slowly erodes the memorisation strategy. The model is, in a sense, under constant silent pressure to find a more efficient explanation. At some tipping point, the generalisation strategy becomes cheaper than the memorisation strategy, and the model flips to it.

The transition is abrupt because it is a phase change, not a gradual shift. Like water turning to ice—nothing, nothing, nothing, and then everything at once.

What This Breaks

Before grokking, every practitioner operated on a simple set of beliefs about training curves. Grokking did not update those beliefs. It inverted them.

None of this means you should train every model for a million steps. The grokking effect was observed on small models doing simple algorithmic tasks with particular training setups. But it does mean the old heuristic—stop when you overfit—may be throwing away models that had further to go.

The Question Nobody Can Fully Answer Yet

Here is where it gets interesting for anyone who works with large language models.

If small models can transition from memorisation to understanding on arithmetic, what are models with hundreds of billions of parameters—trained on the entire internet, for months—actually doing inside their weights? When a large language model gives you a surprisingly insightful answer, is it recalling something from its training data? Or has it grokked the underlying structure of the idea?

We do not have a clean way to tell from the outside. Both strategies can produce identical outputs on familiar inputs. The difference only shows up at the edges—on questions the model has genuinely never encountered in any form.

• A model that memorised will fail confidently when the distribution shifts even slightly.
• A model that grokked will generalise to genuinely novel inputs, including ones its training data could never have anticipated.
• From the outside, in normal operation, both look identical.

Evaluation tells you what a model knows. Grokking asks whether it understands.

Final Thoughts

The paper was titled, with characteristic academic understatement, "Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets." The finding is small on the surface: a tiny model, doing toy maths, trained too long by accident. The implication underneath is not small at all.

It tells us that the line between memorisation and understanding is not fixed. It is not drawn at the moment of overfitting. A model can cross it much, much later—if you let it.

The next time you are staring at a flat validation curve and thinking about killing the training run, ask yourself one question: am I looking at the end of learning, or just the middle of it?

Sometimes patience is not procrastination. Sometimes it is the whole answer.