Store less. Reconstruct more.
Dimensional Multiplexing started from a systems constraint rather than a compression target. Modern models can become memory-bound long before the available arithmetic is exhausted. If useful weight states share enough structure, some of that persistent state may be replaceable with reconstruction compute.
The project established that the construction is mechanically real. Address-selected transforms create distinct logical weights, the byte accounting is exact, and the representation can compose structurally with quantization. On one balanced synthetic composition task it matched a dense INT4 model using about four times its storage.
The general language-model claim did not survive. Across controlled transformer tests, pretrained compression and mechanism probes, the derived logical width behaves primarily like the number of stored degrees of freedom rather than a larger independently learned model.
The structural compression is real. The general language capability advantage is not.
A smaller stored base, expanded at use.
The core construction derives multiple logical weight planes from shared stored state:
Shared stored base state.
Cheap address-selected transform used to derive a plane.
Optional plane-specific residual state restoring local individuality.
The controlled transformer changes only the two-matrix GELU FFN. Attention, embeddings, RoPE, RMSNorm, residual paths and the language head remain conventional. The tested line uses eight all-active planes, two shared bases, learned Givens butterflies, optional two-sided low-rank residuals and INT4 QAT.
The language result is negative under the comparison that matters.
The primary functional comparison is against a measured dense INT4 frontier at equal deployed bytes. Matched training tokens and matched tokens per stored parameter remain separate comparison classes.
20M controlled transformer
| Arm | Deployed bytes | Test loss |
|---|---|---|
| Dense INT4 / FFN 1024 | 12.475 MB | 3.33446 |
| Dense INT4 / FFN 216 / equal-byte control | 7.309 MB | 3.59289 |
| Dimensional INT4 / 10% residual / uniform | 7.347 MB | 3.62284 |
The dimensional arm was 0.02995 nats worse despite being 0.5% larger and doing 54% more arithmetic. At matched training tokens the same comparison was +0.01752 nats. Both comparison classes agree: logical width did not buy more capability than storing a narrower dense FFN.
At 50M, the tight equal-byte pair remained unresolved and the storage multiplier was bounded at no more than 0.996x because the result fell outside the measured dense frontier. There is no advantage to trend. Across the eight-cell residual-by-mixer sweep, more individuality improved absolute loss while capability per byte fell from 1.000x to 0.942x.
Pretrained conversion
The representation was also tested as a way to compress an existing model. Under the same fitting and recovery treatment, the frontier for SmolLM2-135M was:
| Route | Compression | Final bits / UTF-8 byte |
|---|---|---|
| INT4 | 1.000x | 0.8961 |
| INT3 | 1.151x | 0.9350 |
| INT2 | 1.357x | 1.0497 |
| Structured pruning | 1.742x | 1.0894 |
| Dimensional sharing | 1.743x | 1.0951 |
Dimensional sharing reaches beyond the practical low-bit quantization frontier, but it does not own that advantage. Structured pruning reaches essentially the same ratio and is marginally better and simpler.
The project produced more than one kind of result.
Exact byte accounting
Stored values, scales, transforms, residuals, metadata, biases and padding are charged. Sharing and quantization compose structurally.
Structured composition can benefit
The strongest bounded small-task result matched a dense INT4 model using about 4.1x its deployed storage. It did not transfer to language.
Local freedom matters
Selective individuality and learned coordinate mixing can help when the task contains compatible structure. The useful rate is task-dependent.
Recovery became much cheaper
Sequential activation calibration, teacher distillation and a cosine schedule reached a better loss after 30M recovery tokens than the previous process reached after 75M.
Reusable engineering results
- Weight-space reconstruction is preferable to expanding activations for this implementation.
- Whole-step integration gates matter more than isolated kernel microbenchmarks.
- CUDA Graph capture, fused AdamW and selected Triton epilogues materially improved the research stack.
- Test and payload gates caught defects that changed scientific conclusions, not merely code quality.
The failure mode is now measured.
Learned FFN neurons are close to incoherent.
Within-layer mean absolute cosine similarity is 0.0397 against a random-direction floor of 0.0333. Across layers it is 0.0348. There is some structure, but not much.
Cheap transforms contain too little information.
A transform only creates a storage advantage if its description is cheaper than directly storing the weights. Richer transforms fit more structure, but their descriptions consume the byte saving. A full rotation costs 1.50x the values of one plane and still leaves 58% relative error.
Encoding the difference does not change the rate problem.
Across seven residual codecs and a fourfold rate range, plane-difference coding produced zero wins in 40 functioning equal-bit-budget comparisons against directly quantizing the wide weights.
Cheap transforms compress because they contain little information. Because they contain little information, they cannot generate the missing independent capability. Rich transforms can describe more structure, but their descriptions consume the storage saving.
Keep the failed questions visible.
The point of retaining falsified and withdrawn avenues is not to make the project look unsuccessful. It is to stop the next iteration from quietly becoming another version of something already tested.
| Claim | Question | State |
|---|---|---|
C-0001 | Address-selected transforms produce distinct functional weights. | Supported |
C-0002 | Address bits create exponential independent capacity. | Falsified |
C-0003 | Greater effective rank implies greater capability. | Falsified |
C-0004 | Logical expansion improves capability per deployed byte over a storage-matched dense control. | Falsified |
C-0005 | Small local individuality can help shared planes. | Supported / task-bounded |
C-0005 | Coordinate mixing is useful. | Supported / task-bounded |
C-0007 | Conditional plane mixing generalises. | Falsified as a general claim |
C-0007 | Sparse MoE routing unlocks the planes. | Falsified as a general route |
C-0008 | Dimensional Multiplexing beats dense INT4 at equal bytes on language. | Falsified |
C-0011 | The advantage improves with scale. | Falsified / mis-posed |
C-0006 | Sharing stacks beneficially with lower-bit quantization. | Falsified functionally |
C-0012 | Sharing beats pruning when converting a trained model. | Falsified |
C-0014 | Sharing along depth escapes the neuron-axis limit. | Falsified |
C-0015 | Plane differences can be encoded more cheaply than the weights. | Falsified as a rate claim |
C-0013 | Better recovery materially improves compressed models. | Supported |
C-0016 | Deployment throughput is proven. | Open |
C-0017 | A novel dimensional mechanism can improve capability per deployed byte outside the tested construction family. | Open |
Full negative / withdrawal register31 records
| Avenue or claim | Disposition | What killed or limited it |
|---|---|---|
| Exponential capacity from address bits | Falsified | Additive span is at most b+1; derived states are correlated. |
| Effective rank as the capability objective | Falsified | Higher-rank compositional model lost to dense. |
| More transform stages as the primary fix | Falsified | Fit improved; trained capability did not. |
| More shared bases | Falsified as plateau fix | Did not remove the storage-matched ceiling. |
| Instantaneous gradient conflict as an allocation rule | Weakened | Conflict exists but did not predict a useful allocation. |
| Longer training as the universal explanation | Falsified | Did not remove the small-task plateau; later recovery gains came from process, not duration alone. |
| Dense-seeded continuation | Falsified | Training-matched identity control explained the gain. |
| Fixed sharing rate everywhere | Falsified | Residual individuality has a task-dependent interior optimum. |
| Per-weight DFC as deployment line | Rejected | Metadata overhead is disproportionate beside INT4. |
| Sparse MoE as general unlock | Falsified | Task-conditional benefit; spectral regression failure; all-active arm stronger. |
| Conditional mixer as universal component | Falsified | Composition win did not transfer; language at zero residual worsened significantly. |
| Mixer compression | Mostly failed | Removed or degraded the useful conditional behavior for little gain. |
| Checkpointed 63-cell transformer sweep | Invalid | Late-binding closure corrupted gradients. |
| Butterfly fit error as capability predictor | Falsified | Clean out-of-sample prediction failed; ordering nearly reversed. |
| Block-diagonal rotation | Rejected | Better fit, faster and smaller, but worse trained loss. |
| Dimensional INT2 | Rejected | Pays sharing and low-bit penalties plus a negative interaction. |
| Residual individuality restores capability per byte on language | Falsified | Absolute loss improves while storage multiplier falls monotonically. |
| Conditional mixing closes the transformer gap | Falsified | No architecture cell crosses 1.0x. |
| Scaling will reveal a latent advantage | Falsified to 50M | Flat trend; at most 0.996x at 50M as a frontier bound. Nothing above 50M is measured. |
| Prune + share stacking | Falsified | Same-byte arrangements remain inside 0.0224 bits/byte. |
| Sharing recovers faster than pruning | Withdrawn | Pruning had not received activation-aware fitting. |
| Sharing is 2.3x less damaging than INT2 | Withdrawn | INT2 had not received the recovery process. |
| Per-layer rank tilt | Falsified as material lever | Large zero-shot screen became +0.0018 after recovery. |
| Depth multiplexing | Falsified | 0.9838x and 41% more arithmetic under exceptionally tight controls. |
| Plane collapse explains parity | Falsified | A 1.8x diversity change produced effectively identical loss. |
| Weight decay explains parity | Falsified | Removing the asymmetric decay still produced parity. |
| Delta is cheaper than W | Falsified as a rate claim | Seven codecs; zero functioning equal-bit-budget wins. |
| Cluster similar neurons to ease sharing | Falsified | Lower reconstruction error, worse functional loss. |
| Orthogonal spread is a free win | Withdrawn | Partial-run positional result did not replicate across grouping conditions. |
| PDACA recurrence/operator-diversity evidence | Quarantined | Unverified, post-hoc precursor evidence; not support for the mechanism. |
| Capability equals distinct learned parameter count | Withdrawn | Where bytes are placed changes their value; individuality can improve loss while worsening loss per byte. |
The project progressively removed its own shortcuts.
Inception
Show that addressed transforms can create distinct functional states, then bound the capacity claim.
Local freedom
Add residual individuality and coordinate transforms; establish that local freedom can help without solving the general ceiling.
Capability per byte
Reset evaluation around whole-model deployed bytes and a measured dense frontier.
Transformer instrument
Build a controlled FFN-only transformer stack, exact accounting and qualified performance path.
Language result
Run equal-byte controls at 20M and 50M. Logical expansion does not beat stored dense capacity.
Pretrained conversion
Compare sharing, pruning and quantization under treatment-matched recovery.
Mechanism limits
Measure incoherence, transform cost, plane diversity and residual coding to explain the ceiling.
New mechanism search
Move away from local variations and test constructions with a genuinely different source of advantage.
The next mechanism has to change the question.
The negative results define requirements for the next constructions; they do not define the end of the project.
Plane-count sweep
Test 4, 8 and 16 planes while holding the stored base and deployed-byte budget constant. The current prediction is a flat capability line with increasing compute. Eight planes is one measured point, not a theorem about every expansion ratio.
Function-optimised neuron assignment
Treat plane construction as an assignment problem rather than assuming contiguous neuron order is neutral. Compare positional, random, clustered, diversity-spread and function-optimised groupings under identical recovery.
Correlated function families
Apply multiplexing where correlation exists before compression rather than forcing arbitrary FFN neurons into a shared representation. Candidate targets include related adapters, task experts, multilingual adaptations and temporal checkpoints.
Adaptive reconstruction
Move beyond a fixed reconstructed tensor. Activation-conditioned generation, recurrent refinement or conditional low-rank updates make a different prediction because additional compute produces an input-dependent function.
Different memory objects
Weights may be the wrong object. KV state and intermediate activations can be more locally correlated and often dominate long-context deployment memory.
Capability beyond mean NLL
Construct tasks with explicit sets of valid hypotheses and measure valid-hypothesis mass, diversity inside the valid set and collapse after evidence arrives. This is a separate measurement programme, not evidence for the architecture.
Fused reconstruct-and-consume inference
A projected 70B INT4 decode model remains bandwidth-bound after reconstruction and suggests a possible throughput gain. This is a model, not a benchmark. No fused kernel or 70B dimensional checkpoint currently proves the deployment claim.
The falsification is broad, not universal.
- From-scratch language training was measured at 20M and 50M only.
- SmolLM2-135M was used for pretrained compression, not an equivalent from-scratch campaign.
- Capability is held-out FineWeb-Edu pretraining loss, not downstream utility.
- The controlled transformer changes the FFN rather than every memory-bearing component.
- The positive small-task composition result remains real even though it did not generalise.
- The plane-count axis and recovered neuron-permutation lead remain open.
- Adaptive reconstruction and different memory objects lie outside the present falsification.
- Large-model storage and decode gains remain structural or projected, not measured deployment results.
Shared bases, cheap address-selected transforms and small charged corrections have not increased language-model capability per deployed byte over conventional alternatives.
The evidence does not justify the broader claim that learned computation can never trade for stored weights.The public record ends at the public repository boundary.
Experiment records, machine-readable results, claim dispositions and supporting artifacts are maintained in the public project repository. Experiment and claim identifiers used here correspond to immutable public records.
E-0011— Transformer mechanism sweepE-0012— Transformer scale probeE-0013— Pretrained compression and recovery comparisonE-0015— Depth multiplexingE-0016— Activation-aware recovery processE-0018— Plane coherence and angle decayE-0019— Plane rate-distortion
Evidence that has not crossed the public repository boundary is described as unpublished and is not referenced by an internal filename.