Performance & Benchmarks
Measured accuracy, calibration, latency, and adaptation behavior across ARCANE models.
Exploratory Tiny Shakespeare run (15k chars, 10 epochs, unequal parameter counts). Treat as a smoke comparison, not a Transformer result.
Tiny Shakespeare Results
| Model | Val Accuracy | Val Loss | Train Time |
|---|---|---|---|
| Traditional Deep LSTM | 9.50% | 6.85 | ~45s |
| Neuromimetic (Standard) | 10.20% | 6.42 | ~58s |
| Hierarchical Resonance | 11.25% | 6.15 | ~95s |
Observed Results and Limitations
- Hierarchical Resonance used ~385K params vs ~195K for the LSTM baseline.
- Accuracy moved from 9.50% to 11.25% on this small character LM; MNIST gaps in the markdown docs are within noise.
- Mechanism unit tests (STE spikes, closed-form resonance, BCM plastic kernel, GSER elastic gate) live in
tests/test_mechanism_correctness.py. - This run does not isolate inference-time resonance. Use the same-checkpoint ablation harness for cycle-count comparisons.
Run the Benchmark
Terminal
python examples/test_hierarchical_resonance_comparison.py