Autopsy of PR #1019's 27M-Parameter GPT

MLP accounts for 6,247 × 10⁻⁶ total sensitivity. All four attention matrices (Q, K, V, Out) together: 1,600 × 10⁻⁶. For mixed-precision GPTQ: allocate more bits to MLP. Layer 10 is most sensitive per-layer (+1,057 × 10⁻⁶).

Condition number alone is a poor predictor of quantization damage. Effective utilization (stable rank / full rank) combined with dimensionality is what matters. This is directly measured, not theoretical.

Layer 7 contributes −4.35 bits/token — more than any other single layer after L0's initial embedding projection (−15.80). Layers 3–5 show increased loss (residual stream becomes less readable). Hypothesis: encoder layers 3–4 are reorganizing representations for the skip connections (enc 3→dec 6, enc 4→dec 5), and decoder layer 5 pays the cost of integrating the enc 4 skip, sacrificing intermediate readability. This is consistent with the architecture but not proven causally. Layer 10 adds only −0.47 bits/token — a candidate for narrowing, pending empirical validation.

Do not pursue calibration tuning. 70% of total loss comes from tokens where P(correct) < 5% — the model is well-calibrated but often wrong. The bottleneck is accuracy, not confidence. Temperature scaling and label smoothing would yield negligible gains.

22 of 88 heads are previous-token heads, concentrated in the encoder (14 vs 8 decoder). Only 2 show induction behavior (L0H5, L3H2) with marginal scores (~0.02). At this scale, the model relies on n-gram statistics, not in-context copying — induction-head-based interventions are not worth pursuing.