Investigation: does the 6-bit core make K4-Core-K6 think longer?

Direction, everywhere on this page except the vLLM per-group table: ratios put K4-Core-K6, or the part of it under test, first (above 1 = it thought longer); differences are K4-Core-K6's side minus the other (below 0 = it was less ready to stop). G4's published reading is stated the other way (BF16 / 6-bit = 0.80); here it is 1.25.

At a glance

Thinking length in free generation

The stop decision on fixed traces

How the results fit together

• The observation shrank on new seeds. The first run was read because its difference stood out (code 1.43x, prose 1.19x). Rerun on the same prompts with new seeds and a frozen rule: code 1.27x, prose 1.04x. A smaller value on new seeds is what that selection predicts; it does not by itself show the difference is absent. Code prompts were above 1 in both runs (4 of 4 each time).
• The routed experts are ruled out at this resolution. 0.99x [0.93, 1.05], the narrowest interval of the three; the total is the core times the experts (1.16 × 0.99 ≈ 1.15). Whatever difference K4-Core-K6 shows goes with its 6-bit core.
• The stop decision itself does not move on fixed GPQA reasoning traces, for the core on both engines and for the experts on CuteAFD. Those traces are reasoning text, where no length difference shows either (Block L reasoning 1.05x [0.85, 1.39]; vLLM G4, finished replies, 1.03x). The two results agree; neither says how the stop decision behaves on code text, which was not scored this way.
• If the core's longer code reasoning is real, these results are consistent with it building up along the reply rather than with a lower readiness to stop at a given point on reasoning text; they do not test that directly. [inferred]
• The two engines' “BF16 core” differ. The vLLM loader serves the BF16 core as BF16; CuteAFD converts it to FP8 at load, which is how it serves K4. The CuteAFD core contrast is therefore 6-bit against FP8-from-BF16.
• The core is also what makes K4-Core-K6 slower on CuteAFD: with its BF16 core converted to FP8, its experts decode at K4's speed (130 against 129 tok/s on code); with the native 6-bit core, 121.

CuteAFD: core or experts?

Free generation: Block L stage L3 (130,048-token cap, 2 in flight, thinking on), the same 4 code and 4 prose prompts as the run that raised the question, two new seed sets; each seed is shared by the three arms. All 96 replies finished. Reasoning tokens are counted by re-tokenizing the saved reasoning text. Frozen rule: a difference is detected when the 95% interval over the 8 prompts excludes 1; the interval over the 32 pairs is secondary. Read: S1-READ.md.

Fixed traces: the 93 traces of the vLLM investigation (token ids unchanged), scored on CuteAFD's bench probe with a cold cache and drafting off; 93,670 rows per arm; the longest trace (95,362 tokens) took 88 s. The thinking-length run's scorer is used unchanged. Read: S2-READ.md.

A note on one measure. “Max p(</think>) on unfinished traces” (the last column of the vLLM per-group table below) falls on the first scored row, the opening of the reasoning span, in 46 of 48 trace-arms on CuteAFD and 180 of 192 on the vLLM fork. That row is the same for every trace of a question, so the measure is mostly a per-question value counted once per trace. On CuteAFD it flagged the experts (one question carries it); with the opening row removed the order of the arms changes from question to question. On the vLLM fork no group met the rule on it. Post hoc: POST-HOC-S2.json.

vLLM fork: core groups on fixed traces (G2)

• Two groups meet the frozen rule on the close measure: the MLA output and MLA q_b projections (+0.017 and +0.014). Read with them: the reference gap they are measured against (+0.005) is itself smaller than launch variation and its interval includes 0, so their shares of it (3.5 and 2.9) divide by a gap indistinguishable from zero; 9 groups × 3 measures make 27 tests, of which about 1.4 would pass at 95% with no effect; no group meets the rule on the other two measures.
• Post hoc, against other bases: against the second 6-bit launch both intervals include 0 (MLA output +0.011 [−0.002, +0.027], MLA q_b +0.008 [−0.001, +0.017]); against the mean of the two launches both exclude 0 (+0.014, +0.011).
• Size: +0.014 in log p is about a 1.4% relative change in p(</think>) at the close (mean p ≈ 0.70, median 0.94).
• The two launches differ because the vLLM fork's multi-chunk prefill is not run-to-run deterministic: more than half of the BF16-core distance on long traces (0.0071) is the size of the distance between launches (0.0040). On the short single-chunk teacher windows the two launches differ by +0.0000007 in KL [−0.0000013, +0.0000023]. CuteAFD's two launches agree bit for bit on the same long traces.

vLLM fork: free generation (G4)

Loops: what the records show

• How late the loops start. The looping investigation's early-stop detector (the last 30,000 characters checked every 10,000; three ratios below 0.10 in a row) stops no finished reply here (61 replies, lowest check 0.221). On the 4 loops it would fire at 1,190K of 1,406K characters, never (1,372K), 730K of 802K characters, 110K of 1,485K characters: the three loops on questions 81 and 13 began late in their replies, the question-12 loop early.
• On the 16 unfinished traces (cut at 32K tokens, mostly before any loop), the BF16 core makes the recorded continuation slightly less likely per token (−0.0015 [−0.0031, −0.0000]), the same size as between two launches of the 6-bit core (−0.0010 [−0.0023, +0.0004]).
• CuteAFD rerun: 0 of 96 replies reached the 130,048-token cap on any arm (code and prose prompts, not the GPQA questions above).

What was tested

• CuteAFD (the engine where the question arose). tpurtell's CuteAFD v4.0.0, unmodified. Three arms (table above) separate the core from the routed experts; the derived checkpoint is a local build, not a producer release. Step 2, the fixed traces, ran first (four scoring launches, including a second K4-Core-K6 launch); step 1, free generation, followed. Protocol frozen before the first request: PROTOCOL.md. The Block L ratio that raised the question and the max-p check are post hoc.
• vLLM fork, arms. One checkpoint, wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K4-Core-K6 (K4 routed experts, 6-bit EXL3 core and head). The 6-bit arm rebuilds its 316 core tensors to BF16 at load (exact arithmetic, one rounding). Each swap arm loads the named group's original tensors from zai-org/GLM-5.3-Flash-BF16 @ f9fed3bb, the checkpoint's own source, instead; every launch logged the expected number of swapped tensors. Same engine, image, loader and kernels on every arm.
• vLLM fork, engine. tpurtell's vLLM fork v0.9.2 release image with a local loader modification (not a tpurtell release; the release does not load this checkpoint): repro/vllm-k6-loader.
• G2, fixed traces. 77 finished and 16 unfinished reasoning traces, each teacher-forced through every arm: 52 from earlier hard-question probes (questions 13, 79, 88, 121, 127; K3.25 and K4 on earlier fork versions, so off-policy here) and 41 generated by the 6-bit arm itself on the seven length-variable questions. Top-64 log-probabilities at sampled rows and at the last 512 rows before each close, plus the exact log-probability of </think> at every stored row. Plus the P-A teacher panel (25 × 2,048 tokens). CuteAFD scored the same traces at the same rows.
• vLLM protocols. G2's reading rule was frozen before the first capture; a second launch of the 6-bit arm was added (Amendment 1) before any capture was read, because the fork's long-prompt prefill varies between launches. G4 was frozen after G2 was read and before its first sample. The loop checks were chosen after G4 was read and are exploratory.

Reproduce and re-analyse

1. Inputs. Checkpoints wrldsuksgo2mars/GLM-5.3-Flash-EXL3-K4-Core-K6 @ 5d3fffe1, brandonmusic/GLM-5.3-Flash-tr3-4bpw @ a5fee929 and the BF16 source zai-org/GLM-5.3-Flash-BF16 @ f9fed3bb; engines tpurtell's CuteAFD v4.0.0 (tpurtell/cuteafd) and vLLM fork v0.9.2 (tpurtell/glm-5.3-flash-ext3-2x-rtx) with the local loader modification.
2. Derived checkpoint. build_derived.py (standard library; rewrites the shards that hold core tensors, hardlinks the rest); check with hash_core.py against core-hash.json and derived-manifest.json.
3. CuteAFD runs. Configs in results/2026-11-03-cuteafd-core-swap/code/configs; the chain chain-cs.sh; drivers results/2026-11-03-cuteafd-core-swap/code/drivers (fixed-trace probes, identity check, Block L with new seeds) and the v4 Block L driver results/2026-11-03-cuteafd-core-swap/code/v4-drivers, which imports block_l.py.
4. Scoring. Reasoning tokens: analyze_l.py; free generation: analyze_s1.py and block_l_posthoc.py (both read samples.csv's fields); fixed traces: score_stop.py, post_hoc_s2.py.
5. Third-party analysis without the run. samples.csv (one row per reply: arm, seed, finish, reasoning tokens, decode speed), s2-probes.csv (one row per probe, with a digest of the per-row logits hashes), the per-trace scores in results/2026-11-03-cuteafd-core-swap/analysis and results/2026-11-02-thinking-length/analysis; every number on this page is in SUMMARY.json or SUMMARY.json.
6. Not published. Reply and reasoning texts (model output), GPQA question text and the fixed traces' token ids (dataset terms), capture and probe dumps (top-64 log-probabilities of model output), and checkpoint weights (rebuild them from the public sources with the script above).

Open questions

• Is the 6-bit core's longer code reasoning on CuteAFD a real effect? Measured: core 1.25x on code with all 4 code prompts above 1, total 1.27x, and 1.43x in the run that raised the question; the frozen interval over 8 prompts includes 1.
• Does the core change the stop decision on code text? Unmeasured: the fixed traces are GPQA reasoning.
• Does the 6-bit core change how often replies loop? Measured: 1 loop in 18 samples of question 12 on the 6-bit core (6 earlier, 12 in G4), 0 in 12 on the BF16 core (Fisher p = 1.0), vLLM fork; on the seven questions, K4-Core-K6 reached the cap about as often as K4 and K3.25 in earlier records (table above). The per-token stop measures do not see a difference. No reply reached the cap in the CuteAFD rerun (96 replies, code and prose).

Caveats

• Two engines, one machine. The vLLM loader rounds the 6-bit core once from exact arithmetic; CuteAFD runs it natively and the producer's own rebuild rounds at FP16 between stages (15.8% of elements differ, 2.4% by more than one BF16 unit). CuteAFD's “BF16 core” is BF16 converted to FP8 at load.
• The routed experts are compared as two whole quantizations (K4-Core-K6's and K4's), not tensor by tensor; the derived checkpoint is a local build, not a producer release.
• Free generation on CuteAFD covers 8 prompts (4 code, 4 prose), 4 replies per prompt per arm; the Block L ratio that raised the question is post hoc. G4 on the vLLM fork is one GPQA question; the knife-edge questions where replies tip between short and capped were dropped from G4 because their spread would need hundreds of samples per arm.
• The fixed traces are GPQA reasoning generated on the vLLM fork, teacher-forced on both engines.
• KL here is a top-64 measure with the arm's 64th value as the floor for tokens outside its top 64 (a lower bound), not full vocabulary. GLM tokenizer throughout; not comparable with other model families.

Data: CuteAFD part results/2026-11-03-cuteafd-core-swap (SUMMARY.json, S1-READ.md, S2-READ.md, protocol/); vLLM part results/2026-11-02-thinking-length (SUMMARY.json, G2-RESULT.md, G4-RESULT.md, LOOP-CHECKS.md, protocol/). Reasoning texts, question text and dumps are not published.