Human Grokking

This paper started with something I noticed in my own learning.

For a long time, learning across mathematics, physics, electrical engineering, and RF felt like collecting things. A definition went here, an equation went there, and understanding meant being able to retrieve the right item when I needed it.

At some point that changed. New ideas started arriving with some of their shape already determined. If I knew what a concept connected to, what it preserved, and what role it had to play, I could often anticipate it before learning its formal name. The experience was less like remembering an answer and more like seeing that only a narrow class of answers could fit.

I wanted to know whether that was merely the ordinary accumulation of expertise or a more specific change in how the material was represented.

Why call it grokking?

In the machine-learning experiments that introduced the term, a model first memorized its training examples while performing poorly on held-out examples. With continued training, its test performance later rose sharply. It had moved from fitting the examples to using a structure that generalized beyond them. Power and colleagues describe the original result in Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets.

Nanda and colleagues later looked inside one of these models. They found that the apparent jump in behavior was built from gradual internal changes: memorization, formation of a generalizing circuit, and cleanup of the memorizing circuit. The output changed suddenly when one mechanism overtook the other; the underlying process was not itself a single discontinuous leap.

That pattern gave me a way to state the human question. A learner might accumulate facts first, gradually form a dense network of relationships beneath them, and eventually begin navigating the relationships instead of retrieving each fact independently. The subjective “click” would be the point where the second mode became easier to use than the first.

This is a proposed parallel, not a claim that a human brain and a small transformer learn by the same mechanism. The observation behind it is my own. It is one person’s report, used to generate a hypothesis.

The constellation graph

I built a graph to make the hypothesis concrete. It contains 32 constellations and 91 connections across physics, mathematics, electrical engineering, and RF engineering. A constellation is a small cluster of related terms, equations, invariants, and connections to neighboring clusters. The edges are labeled with relations such as limit, conservation, duality, transform, embedding, approximation, and cross-domain bridge.

The point is to present knowledge as a connected structure rather than a sequence of isolated topics. If a learner can use enough of those connections, a missing concept should become partly inferable from the surrounding constraints.

The graph is a curriculum prototype and a way to generate test questions. It is not evidence that the proposed transition occurs. I authored the nodes and edges, so their existence shows how I organized the material, not how another learner represents it.

The word morphism also needs restraint here. In category theory it has a precise meaning. In the graph I use it more broadly for a typed structural relationship. Some connections can be written as genuine structure-preserving maps. Others are approximations, limiting cases, or useful analogies. They should not all be promoted to mathematical identities simply because the graph gives them an edge.

The circuit example is one of the firmer connections. A circuit’s incidence matrix is the boundary map from edges to nodes. Kirchhoff’s current law places valid currents in the graph’s cycle space, while voltage differences derived from node potentials live in the corresponding coboundary space. The same chain and cochain machinery describes both the graph and the circuit constraints. That is a real mathematical correspondence. It does not make every cross-domain resemblance equally exact.

Conversation as pressure

The learning that prompted this paper happened through sustained collaborative conversation, including work with AI systems. Explaining an idea to someone else forces compression. Questions expose gaps that can remain hidden in private thought. Repeated exchange also makes inconsistent uses of the same concept harder to ignore.

The paper compares that pressure to regularization in machine learning. I still think the analogy is useful, but it is an analogy. Weight decay is a defined operation on model parameters. Conversation is not a known cognitive equivalent of it, and I have not shown that stronger conversation causes a grokking transition.

The smaller claim is enough to test: a learning process that repeatedly demands explanation, compression, and application to new cases may produce better relational understanding than passive exposure to the same material.

What would count as evidence

The paper proposes five ways to test the framework:

1. Remove a concept from part of the graph and test whether the learner can reconstruct it from the remaining relationships better than a guessing or surface-association baseline.
2. Track whether explanations shift from “here is the definition, therefore…” toward “these constraints must hold, so the concept has to be…”
3. Compare learning speed in a new domain that shares known structure with the learned material against a matched domain that does not.
4. Follow individual learners over time and look for the predicted plateau and later jump instead of smoothing everyone into a group average.
5. Test transfer to a structurally related domain that was held out of the curriculum entirely.

These are proposed experiments, not results. The thresholds, control conditions, coding rules, and analysis would need to be fixed before collecting data. If learning improves smoothly, if the graph adds no benefit over an ordinary curriculum, or if the supposed transfer disappears under matched controls, the stronger version of the idea fails.

I do not need the phase-transition claim to be true for the graph to remain useful. It gives me a concrete way to teach and test relationships instead of only definitions. The open question is whether relational density does more than improve learning gradually—whether it can change the mode in which a person understands.

Until that has been measured, human grokking is a name for a first-person experience and the research program that came out of it.

The paper is archived on Zenodo.