🚨 Analysis: “Can a Language Model Learn Facts Continually in Its Weights?”, Baseten’s Latest Paper They sequentially force-fed 247 invented facts into Qwen3 weights, watched most of them become unreachable after a few dozen later writes, and still framed the whole exercise as “continual learning of facts.” What they actually demonstrated is that fine-tuning is cosmetic on a probability landscape whose mass is still dominated by pre-training. [ What the study says ] Bare-statement writes give near-perfect recitation that collapses under any real use. Study-style data does better but still loses most of its reach after twenty sequential updates. “Forgotten” facts keep the majority of their original log-probability lift. Put the same statement back in context and accuracy jumps up to 80%. Two written facts almost never work together. Self-retrieval of anything the model was just trained on sits at 34%. Context barely degrades while the weights tear themselves apart. Full paper:
arxiv.org/abs/2607.11020 [ What the transformer actually does ] It samples from a conditional distribution whose overwhelming mass was fixed in pre-training. A fine-tune is a superficial local edit. It can temporarily raise the probability of a target pattern under a narrow set of queries, but it does not remove the underlying density. The pre-training prior eventually reasserts itself. Every later write is another tear: local overfitting that redirects query geometry, produces interference, and leaves the bulk of the original distribution intact. That is why the “forgotten” fact is still sitting there in log-prob space and why context restores it instantly. Context never tore the landscape; the weight updates did. [ Exhibits ] • Weight writes create question-keyed patches, not addressable facts (a keyhole)
• Later writes do not erase the earlier probability mass; they simply redirect the queries that used to reach it
• 70% of failures under bare-statement training just spit out the newest patch instead
• Context recovers a “lost” study fact up to 80% with zero weight change
• Joint use of two written facts collapses; the model cannot list its own recent training
• Capability damage tracks KL divergence from the original model, every edit costs global coherence All of it is the pre-training distribution reasserting dominance over superficial local bias. Weights are not a reliable knowledge substrate. [ Conclusion ] This is not a partial success of continual learning. This is a controlled demonstration that fine-tuning is mostly BS for durable knowledge. You are not writing facts into the model. You are temporarily scarring the surface of a distribution whose mass still belongs to pre-training. The scars fade or get overwritten; the original landscape wins. Continual learning is the latest industry chimera, built on a category error that projects human traits like memory, knowledge and cognition onto a system whose every output is merely an isolated, local, query-constrained rollout. [ The real problem ] We keep treating weight updates as if they were knowledge installation. They are not. They are high-risk cosmetic interventions that introduce tears, interference, and side-effects while the pre-training prior remains the default. ICL and context work better precisely because they leave that prior untouched. [ What actually works ] Stop pretending fine-tunes install durable facts. Put the information in context when it must remain reachable. Treat every weight edit as a temporary, lossy patch with measurable global damage. Verify the sampled output against the actual distribution, not against the fantasy that you rewrote the model’s knowledge. The knowledge store was never there. The fine-tune did not create one. It just tore the landscape a little more. Apply at your own risk. ☕️ How many more papers are we going to publish that treat superficial weight edits as if they were permanent knowledge updates?

> Me : Think again. More patiently. > Claude: Flagged . . .

The block principle is the basis of the brain's cognition of everything in the world. It is as important as living things are made of cells. Block principle: The brain's cognition of things consists of neurons that record sound, words and physical images respectively. When any part of this block is triggered, all neurons in the whole block will be activated. A language composed of sound and words is used to bind physical images. When you hear the pronunciation, you will activate the neurons of both the text and the physical image; When you see words, you will activate neurons of both pronunciation and physical images; When you see a physical image, you will also activate both sound and text neurons. This is the basis of the brain's cognitive composition of all things in nature. Take the object "Sun" as an example. When you hear the pronunciation of the word "Sun", it is bound to the object and words, and the brain perceives the corresponding words and physical images. When you see the words "the sun", although it is not read, its pronunciation and physical image will appear in your mind; Similarly, when you see the sun, you know its pronunciation and words in your mind. This is because when the word "sun" is converted into EEG signal by vision, the brain's EEG passes through the neurons of its pronunciation and still image at the same time, and the pronunciation of the text and its object image appear in consciousness. So no matter which part is triggered, the text, sound and physical image of the whole block will appear in the perception system at the same time. 3D animation is based on the concept of planarization, which helps to better understand the concept of block principle. In fact, the projection of any complete object by the brain requires multi-dimensional and a large number of brain regions to participate, and neurons that record words, sounds and physical images are not piled up in the same area. Just like when we look down at the earth from a panoramic perspective in space, it is a blue plane sphere, but in fact it is multidimensional and colorful. When studying difficult things, we must simplify and conceptualize the whole universe, the most complex organ, in order to deduce the movement mode of EEG between neurons from a macro perspective through thought experiments without any instruments and data. Simplicity is the symbol of genius-Leonardo da Vinci The goal of science is simplification, not complexity-Richard Feynman. The simplest theory is often the most powerful-Stephen Hawking.