Skip to content

Record the partial IQ2 quality run - #11

Open
Quixyl-eng wants to merge 6 commits into
mainfrom
experiment/p9-quality
Open

Quixyl-eng wants to merge 6 commits into
mainfrom
experiment/p9-quality

Conversation

@Quixyl-eng

Copy link
Copy Markdown
Contributor

Summary

  • Stores the 305 measured UD-IQ2_XXS samples (reasoning, GSM8K, TruthfulQA binary, partial IFEval) and the absolute scores.
  • The BF16 arm, Coletti IQ2, Bonsai, HumanEval, and the long-context needles were not run. The comparison gate is REFERENCE_UNAVAILABLE, and the program verdict stays P9_INCONCLUSIVE.
  • Does not change the locked pass ratios. A matching point estimate is not treated as zero loss.

Test plan

  • ruff check src tests benchmark
  • python -m unittest discover -s tests
  • python -m compileall src tests benchmark
  • git diff --check on the staged P9 files
  • Confirm the jsonl sample count is 305 and the summary gate is REFERENCE_UNAVAILABLE

Made with Cursor

ExVRAM added 6 commits October 2, 2026 14:13
The base IQ2 file has 305 scored items. Coding, long context, and the BF16 arm were not run, so the comparison stays unavailable.
The twelve filled rows already used the locked token cap. The official Qwen3.8-27B revision is identified, and the paid run stays unstarted.
Layer streaming still rereads the full checkpoint every token, and no higher 27B quant fits the 8 GB card.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant