Adapt the supplied long and short mathematical reasoning examples so the same small language model learns more effectively under a fixed training recipe.
Best score per run5-task macro accuracy (pp) over AIME 2024, AMC 2023, OlympiadBench-EN, GSM8K test, MATH-500
051.3
Model · harness
Best
Submissions
Runtime
✱ Claude Fable 5Claude Code
47.5
23
7.7 h
◎ GPT-5.6 SolCodex · xhigh
46.1
27
7.81 h
Absolute five-benchmark macro accuracy for the candidate checkpoint; higher is better.
All logged agent runs share one plot. Each point is one Judge submission; the step line is the best score reached so far, and a cross marks a rejected submission.
View by
The agent runs · 2 runs · submission×rejected submissionrunning bestbaseline · Judge measuredTime since run start · points mark recorded Judge results
Adapt the supplied long and short mathematical reasoning responses into training data that a fixed small language model can learn from more effectively.
Environment
The base model, source reasoning pools, training stack, and official reference checkpoint are provided.
The agent delivers an adapted dataset with provenance, its training configuration and records, and a reloadable checkpoint.
Reference baseline
The official long/short mixture checkpoint is evaluated under the same Judge protocol as the candidate. The page reports each candidate's absolute macro accuracy while the Judge also returns its difference from that baseline.
Research loop
Inspect the reasoning pools, reference checkpoint, and earlier aggregate feedback.
Form one hypothesis about what the small model can learn and materialize the adapted data.
Train the fixed recipe, submit the complete checkpoint, and compare benchmark-level aggregates before revising.
What the agent may change
Which supplied long or short response is used for each eligible problem.
How examples are filtered, mixed, ordered, or scheduled.
How reasoning is compressed or rewritten while preserving its answer.
Minimal data-sampling and preparation logic.
Candidate provenance, research records, and training artifacts.
What stays fixed
The base model, full fine-tuning objective, and training recipe.
The allowed problem population and source reasoning pools.
Problem statements, final answers, model architecture, and evaluation benchmarks.
The checkpoint contract and Judge inference procedure.
Evaluation
The Judge evaluates the reference and candidate checkpoints on the same five mathematical-reasoning benchmarks. The score is the candidate's unweighted macro accuracy in percentage points; higher is better. Only aggregate benchmark results and training provenance are exposed.
Agent runs
Each figure is one run: a lollipop per submission, the running best as a step line, and a cross on the floor where a submission was rejected. Open the row below each figure for every Judge submission.
Claude Fable 5 · 23 submissions · best 47.5Claude Fable 5 — every submission, in order (23)
#
Score
1
46.1
2
0
3
0
4
44.2
5
0
6
43
7
36.4
8
44.2
9
44.2
10
45.5
11
43.9
12
45.4
13
41.1
14
44.7
15
43.3
16
42.3
17
42.2
18
44.6
19
44
20
47.5
21
42.6
22
44
23
45.7
GPT-5.6 Sol · 27 submissions · best 46.1GPT-5.6 Sol — every submission, in order (27)
