decode-path perplexity, teacher-forced, 512 tokens of docs/source/en/philosophy.md, 3-bit experts
bf16 dense  3.3558
8-bit dense (before the hyper-connection fusion) 3.3642
8-bit dense (current) 3.3629, top-1 agreement with bf16 dense 94.5%, with the pre-fusion 8-bit 96.7%