BIM: The Full Hypothesis
Transformers are frozen. After training ends, their weights do not move. Information you give them in a conversation persists only in the context window — a temporary buffer. When the session ends, it is gone. They cannot learn from a correction. They cannot remember you. They are, in the most literal sense, static.
BIM is our attempt to build something that is not.
The structural problems with the dominant approach
Three things make transformers unsuitable as autonomous adaptive agents — not as tools, but as agents that perceive, act, and grow inside an environment:
- No plasticity at inference time. Weights are fixed. You cannot teach a deployed model anything. What it knows is what it knew when training stopped.
- Catastrophic forgetting under fine-tuning. Dense networks share weights across everything they know. Fine-tuning on new data overwrites old knowledge. You cannot add without subtracting.
- A power wall. Running and adapting these systems requires GPU clusters consuming kilowatts. The biological brain performs continuous multi-modal learning on roughly twenty watts.
The hypothesis
A small, sparse, locally-learning neural substrate — modelled directly on principles from neuroscience — can deliver capabilities that transformers cannot, even at radically smaller scale:
- Online learning from individual interactions
- Episodic recall by pattern completion
- Continuous adaptation without catastrophic forgetting
The mechanism that makes this possible is sparsity. When only 0.39% of neurons are active at any step, memories become orthogonal to one another — learning a new pattern does not disturb old ones, because it activates an almost entirely different set of cells.
How BIM works
Each input maps to a Sparse Distributed Representation (SDR): a 16,384-dimensional vector with exactly 64 active bits. The choice of 64 out of 16,384 is not arbitrary — it produces the mathematical properties needed for both overlap detection and memory isolation.
Learning is gated by surprise. At each step, the cortex predicts what SDR comes next. The prediction is compared to reality using Jaccard distance:
surprise = 1 − |predicted ∩ actual| / |predicted ∪ actual|
If surprise is zero — you predicted correctly — no synaptic update occurs. If surprise is high — you were wrong — plasticity fires fast. This prevents the system from wasting learning capacity on what it already knows.
High-surprise events are bound into a CA3-inspired hippocampal attractor. Later, a corrupted or partial version of that memory — even a 50% corrupted cue — is enough to trigger full retrieval by pattern completion. This is episodic memory implemented in synaptic weights, not an external database.
Four generations
BIM 0 (2026). A directed graph with Hebbian edge strengthening. Proved that a blank-slate system could acquire associative structure from interaction. The graph grew super-linearly — it was not scalable. But the principle held.
BIM 1 (2026). Replaced the graph with a sparse cortex. SDRs at 16,384 dimensions, 64 active bits. Numba JIT-compiled kernels: 160 tokens per second on a single CPU core. 100% one-shot recall on 20-token sequences.
BIM 2 (2026). Surprise-scaled plasticity. Jaccard prediction error gates learning. Added a second temporal-pool layer and synaptic homeostasis — saturated permanences are rescaled and pruned to keep the system functional over time.
BIM 3 (current). Unified substrate. All five biological modules — sensory codec, predictive cortex, neuromodulator, synaptic homeostasis, CA3 hippocampus — cooperating under local learning rules. No backpropagation. No external memory. No tokenizer. One coherent network.
The failure we had to document
Between BIM 2 and the unified BIM 3, we built an intermediate system that augmented the biological core with classical engineering components: byte-pair encoding tokenisation, an external episodic memory store, a symbolic concept graph, and sharded multi-process weight merging.
It failed on every evaluation gate after 15 hours of training.
Three root causes: synaptic saturation from high-frequency tokens, a silent top-down apical projection (bias clamped to a range that never reached the activation threshold), and reversed winner-cell scoring. But the deeper lesson was architectural — biological substrates do not compose cleanly with engineering patterns. You cannot bolt symbolic reasoning onto a Hebbian cortex and expect the combination to behave like either.
The unified substrate was the answer: everything lives as activation patterns and synaptic weights in one coherent network. No seams.
What we are building toward
Not a tool. A substrate for digital species — localised agents that perceive, act, grow, and learn dynamically inside their environment. Agents with autobiographical memory, genuine personality that emerges from their interaction history, and the ability to learn from a single correction without forgetting everything they knew before.
The transformer is an extraordinary achievement. It is also a dead end for this particular goal. BIM is the attempt to find a different path.