The problem I set out on was catastrophic forgetting. Teach a model a new task and it tends to wipe out what it knew before. I wanted a setup where a model could pick up tasks one after another and keep the old skills intact, and I wanted to test it honestly rather than on a toy benchmark that hides the failure.
The first version, v1, was ambitious. The idea was a central JEPA core acting as a kind of shared brain, with per-task LoRA adapters hanging off it and a couple of auxiliary losses, an LLM-JEPA cosine term and a SIGReg regulariser, nudging the representations to stay well conditioned. I built it in phases so each piece could be checked on its own.
Phase 0 was a sanity baseline on GSM8K, and it came in at 60.8 percent. Phase 1 was the one I was proud of. Across two tasks the model showed forgetting of basically zero and an average accuracy of 0.852. That is the whole pitch of continual learning working in miniature: learn the second task, lose nothing on the first.
Phase 2 is where it broke. I added a gate that was supposed to decide when an update was safe to keep, using a probe on the hidden states. The gate failed. The delta came out at minus 0.5 points and the hidden states collapsed, the same condition-number warning sign I'd learned to watch for. I wrote the full diagnosis into the repo rather than quietly deleting the run, because the failure was the most useful thing in the project. A gate that approves a harmful edit is worse than no gate.
So in June I pivoted instead of patching. v2 throws out the central-brain framing. The new shape is a frozen core, an external memory that holds what's been learned, and a probe-gated rollback that can undo an update when the probe says the representation degraded. Freezing the core is the key move. If the part everything depends on never drifts, a bad task update can't poison it, and rollback becomes a real option instead of wishful thinking. The v1 work is archived under legacy in the repo, kept as reference, not maintained.
Two things I took away. First, zero forgetting is reachable, Phase 1 proved that much. Second, the hard part isn't learning the new task, it's deciding what to trust, and a probe that gates updates has to be more reliable than the updates it's judging. v2 is built around that second lesson.