Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
FOUNDRY_PROJECT_ENDPOINT=
FOUNDRY_BASE_MODEL=
FOUNDRY_MATH_BASELINE_DEPLOYMENT=
FOUNDRY_MATH_FINE_TUNED_DEPLOYMENT=
FOUNDRY_EXISTING_MATH_RFT_JOB_ID=
Expand Down
12 changes: 6 additions & 6 deletions managed_fine_tuning/03-core-rft/02-rft-math-reasoning/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,19 @@
**Level:** Advanced / quota-dependent
**Canonical notebook:** [`notebooks/demo.ipynb`](notebooks/demo.ipynb)

This notebook preserves the OpenR1-Math-220k subsets and the custom boxed-answer
grader while making the grader contract, adversarial tests, bounded monitoring,
and validation reward history explicit.
This notebook uses [OpenR1-Math-220k subsets](data/README.md)
and a custom boxed-answer grader. It includes adversarial grader tests,
bounded monitoring, and validation reward history.

## Safe execution

Copy this module's `.env.template` to a module-local `.env` and configure your own project endpoint and base model. Offline integrity and grader tests require no Azure resources. Executing the upload/training cell submits or resumes a paid job. The existing-job option is consulted when that cell runs. Reuse accepts pending, validating, queued, running, or succeeded jobs only when their model, input file IDs, and reinforcement recipe match. A mismatched, failed, or cancelled job raises an error rather than submitting a replacement.
The training model is `qwen3.6-35b-a3b`, defined by `MODEL` in the notebook. Copy this module's `.env.template` to a module-local `.env` and configure your own project endpoint. Confirm model access, region support, and quota before training. Offline integrity and grader tests require no Azure resources. Executing the upload/training cell submits or resumes a paid job. The existing-job option is consulted when that cell runs. Reuse accepts pending, validating, queued, running, or succeeded jobs only when their model, input file IDs, and reinforcement recipe match. A mismatched, failed, or cancelled job raises an error rather than submitting a replacement.

Before interpreting a completed job as equivalent, also verify training and validation file sizes and agreement between service scores and local regrading. [`live_validation.py`](live_validation.py) is a reusable contract-checking library, not an automation runner. It provides these additional checks and parses the reward and loss columns from the downloaded result CSV; the notebook's recipe check does not automatically perform local/service regrade parity.

Job-creation POSTs are submitted once, with SDK retries disabled. Reconcile an ambiguous submission failure in Foundry before retrying. Read and polling operations may retry transient errors. The separately guarded advanced-math baseline and endpoint-evaluation methodology is retained.
Job-creation POSTs are submitted once, with SDK retries disabled. Reconcile an ambiguous submission failure in Foundry before retrying. Read and polling operations may retry transient errors. Baseline and endpoint evaluation are optional and separately enabled.

The representative source run completed training and produced validation reward
history `[0.05, 0.2]`.

See [`assets/charts/representative-validation-reward.svg`](assets/charts/representative-validation-reward.svg) and [`assets/metrics/representative-run.json`](assets/metrics/representative-run.json). These values are evidence from one run, not guaranteed outcomes; model availability, quota, data sampling, grader behavior, and service updates can change results.
See [`assets/charts/representative-validation-reward.svg`](assets/charts/representative-validation-reward.svg) and [`assets/metrics/representative-run.json`](assets/metrics/representative-run.json). These assets are historical evidence, not a benchmark for the bundled dataset or guaranteed outcomes; model availability, quota, data sampling, grader behavior, and service updates can change results.
Original file line number Diff line number Diff line change
@@ -1,10 +1,20 @@
# Advanced math RFT data

These preserve the exact existing curated OpenR1-Math-220k records:
The notebook uses 100 training and 10 validation records under [preserved](preserved/), with separate, non-overlapping problem prompts.

- `openr1-math-training-rft-100.jsonl`: 100 training prompts and reference answers.
- `openr1-math-validation-rft-10.jsonl`: 10 validation prompts and reference answers.
The source dataset is [OpenR1-Math-220k](https://huggingface.co/datasets/open-r1/OpenR1-Math-220k/tree/e4e141ec9dea9f8326f4d347be56105859b2bd68), revision `e4e141ec9dea9f8326f4d347be56105859b2bd68`, `default/train` (Apache-2.0). The [manifest](hashes.json) records provenance, row counts, canonical-LF byte counts, and SHA-256 integrity values.

Reference answers include both equation roots (training 8), all four real pairs (60), both side-length branches (90; discard zero when `a=b`), and both sphere radii (92). Training 58 uses real parameters with complex roots allowed, yielding `p = ±2^(1/8)`; training 17 uses integers `k ≥ 1`. Training 71 specifies one valid magic square; the text grader does not recognize equivalent rotations/reflections. Multipart prompts use the final-subproblem reference convention.

Hash, schema, split, and boxed-answer checks do not prove every answer is mathematically correct or every equivalent expression receives credit.

## Dataset files

These are the notebook's upload inputs:

- [Training data](preserved/openr1-math-training-rft-100.jsonl): 100 training prompts and reference answers.
- [Validation data](preserved/openr1-math-validation-rft-10.jsonl): 10 validation prompts and reference answers.

Each record contains user messages plus an `answer` field used by the grader. It intentionally has no assistant message: reinforcement fine-tuning explores responses and receives reward rather than imitating a stored reasoning trace.

See `hashes.json` for source mapping and canonical LF byte counts and SHA-256 integrity values. Checks normalize checkout line endings without rewriting or changing any dataset record.
Integrity checks normalize checkout line endings without modifying dataset records.
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,24 @@
{
"path": "data/preserved/openr1-math-training-rft-100.jsonl",
"source": "Demos/RFT_Math_Reasoning/training_rft.jsonl",
"bytes": 54807,
"bytes": 55253,
"rows": 100,
"sha256": "c6b7b4c86b21c796780b8d2fedcfe58433fbeacef77546522c19b8635f6a4184"
"sha256": "37e5573f18ea12e4be1d423deabfd4036c282fb935c28e61a29e9f4351f60b02"
},
{
"path": "data/preserved/openr1-math-validation-rft-10.jsonl",
"source": "Demos/RFT_Math_Reasoning/validation_rft.jsonl",
"bytes": 4868,
"bytes": 4921,
"rows": 10,
"sha256": "fa953157b93d657506a2c3f6d02ff7abeb752c0488b8677ba2167d534e182f28"
"sha256": "beabdabc9ec3bf5509f03cb9a7e93cef6f068b9c0fbc970a105910b3de97b4dd"
}
],
"normalization": "CRLF-to-LF"
"normalization": "CRLF-to-LF",
"source_dataset": "open-r1/OpenR1-Math-220k",
"source_revision": "e4e141ec9dea9f8326f4d347be56105859b2bd68",
"license": "Apache-2.0",
"corrected_rows": {
"training": [2, 3, 8, 17, 23, 25, 27, 32, 33, 35, 58, 60, 64, 69, 71, 77, 78, 80, 81, 83, 90, 92, 99],
"validation": [1, 5, 6, 7, 8, 9]
}
}
Loading
Loading