Essays · Founder think
Software vocabulary was built for machines that repeat themselves. AI doesn't, and "emergence" is the word that tells us how to build, test, and own systems whose behavior comes from use.
Reviewed by Marvin Danig on 2026-10-23. Report issues to editor@achiral.ai.
Emergence Is Not a Bug

For most of my career, a wrong answer from software meant a bug. You could reproduce it, find the line that caused it, fix that line, and the wrong answer went away for good. The whole craft rested on one assumption: the same input gives the same output.
Ask an AI system the same question twice and you can get two different answers. Often neither of them is a bug.
I think our vocabulary hasn't caught up with that. We still describe AI systems with words that were made for machines that repeat themselves: store, record, fetch, spec, bug. Those words aren't wrong. They're just built for a world where behavior lives inside a component you can point to. In a non-deterministic system, a lot of the behavior lives between the components. There is an old word for that, and I think it's the most useful word we have right now. The word is emergence.
A word from 1875
The philosopher George Henry Lewes introduced "emergent" in 1875, in the second volume of Problems of Life and Mind. He needed it to separate two kinds of effects. Some effects are resultants: you can calculate them by adding up their causes, the way two forces combine into one. Others are emergents: they're new in kind, and you can't get them by adding up the parts (Internet Encyclopedia of Philosophy).
A century later, the physicist Philip Anderson made the same argument for science as a whole in "More Is Different" (1972). Knowing the laws that govern the parts doesn't let you rebuild the whole. At each new level of complexity, new properties appear, and each level needs its own science. Anderson never uses the word "emergence" in that essay. He didn't need to.
So the word wasn't made for poetry. It was made for a practical problem: what do you do when the behavior you care about isn't in any of the pieces? That's the problem anyone building with AI has today.
The word AI already borrowed
AI research picked up the word a few years ago, and the story of how it did is worth knowing.
In 2022, Wei and colleagues described "emergent abilities" in large language models: abilities that are absent in smaller models and present in larger ones, with performance near random until the model passes some scale. It was an exciting idea. Scale up, and new skills switch on.
A year later, Schaeffer, Miranda and Koyejo asked whether those jumps were real. They showed that, for the abilities they studied, the jump depended on how the answers were scored. Score with exact match, where a single wrong token makes the whole answer wrong, and you see a sudden leap. Score the same outputs with a continuous measure, and the improvement is smooth and predictable.
I don't read that as a reason to drop the word. I read it as a warning about how to use it. A claim of emergence is only as good as the measurement behind it. If you want to say a system does something new, you have to measure it in a way that could have shown you it didn't.
What the mathematics says
Emergence can sound mystical. It isn't. There is no single equation for it, but there are a few ideas that make it concrete, and three of them matter for AI.
The first is that order can show up at the level of the group even when every single event is uncertain. Flip one fair coin and you can't say what you'll get. Flip it many times and the average settles:
No single flip knows the ratio. The ratio still holds. That's the right way to think about a non-deterministic system. You can't promise what one answer will be. You can say a lot about the distribution of answers.
The second is that behavior lives in the update rule. The logistic map is a tiny equation:
For some values of , the system settles on one value. Nudge and it starts to oscillate. Nudge it further and it becomes chaotic. Nothing about any single changed. The rule changed, and the whole system changed character.
The third is what Stephen Wolfram calls computational irreducibility. Some systems have rules simple enough to write on a napkin, and still there's no shortcut to knowing what they'll do. You have to let them run.
Put those together and you get a fairly sober definition. An emergent system is one whose behavior you can describe well as a distribution, whose character depends on its update rules more than on any one part, and whose future you often can't skip ahead to. That describes a modern AI system quite well.
What the word changes
If emergence is the right word, it changes three things: how we build, how we test, and who is responsible.
Building. When behavior emerges, you don't design the output. You design the rules that produce it. This is the choice we made at Achiral. Our memory is inspired by ACT-R, and it leans on what ACT-R calls the sub-symbolic layer. Every memory has an activation that rises when the team uses it and decays with time. A nightly consolidation pass sets aside memories that stop being used. They go dormant, and they aren't deleted. No hand-written rule decides what a team should remember. Use does. One caveat on where this stands: we still set some of the rates by hand, such as how fast each tier of memory decays. The rules are ours. What the memory becomes is the team's.
Testing. You can't test an emergent system with one assertion. If the right answer can change next month, because the team's use changed, then "it returned the same record" stops being the test. You have to ask different questions. When the system is confident, is it right about as often as it claims? Run the same history twice, in two independent copies, and how often does the outcome change? Does private memory stay private, however the rankings move? That's why we built Cognoscenti, an open benchmark that replays the same seeded team history through the same public API, scores calibration, stability, and scope, and reports how often an answer flips across identical runs. And it's why the mirage paper matters to me. If our measurement can't fail, our claims mean nothing.
Responsibility. If a system's behavior emerges from how a team uses it, then part of that behavior belongs to the team. Nobody wrote it, and no vendor shipped it. It formed through work. That raises a hard question: who owns it? My view is that personal memories belong to people and work memories belong to the business that uses them. And if memory is formed by use, it has to be something you can inspect, correct, and take with you. That's why we drafted GEMs, an open format for memory that emerges this way.
A team's assistant, a few months in
Here is what emergence looks like on a real team, in a scenario I'll make up but that I think most teams will recognize.
An assistant has worked with an engineering team for a few months. Nobody configured it to prefer small releases. But the team rolled back two large launches, talked about why, and has shipped in small steps ever since. Now, when someone asks it to plan a release, small steps come up first.
That behavior is in no single message, ticket, or document. It emerged from the team's own history.
This is the good side of emergence, and it's what teams actually want from AI: an assistant that gets better at their work because it has been part of it. The other side is just as real. A team won't trust behavior it can't see, measure, or correct. So the word comes with obligations. If you build a system that changes with use, you owe the people using it a way to see what changed, why it changed, and how to change it back.
The humility in the word
What I like most about "emergence" is that it's an honest word. It admits that the people who build these systems don't fully control what they become. We choose the rules. We choose what to reinforce, what to let fade, and what to measure. Then the system runs, alongside the people who use it, and something forms that none of us wrote down.
I find that exciting, and a little humbling. For deterministic software, the job was to make the machine do exactly what we said. For AI, I think the job is to shape the rules well, measure what comes out with tests that can fail, and leave the people who use the system in charge of what it becomes.
A monthly trip down memory lane
Subscribe to our newsletter on memory, intelligence, emergence, life, and the pursuit of being less wrong. Handwritten. Once in a month.