Curriculum-Guided Adaptive Recursion Cuts Training Time for Recursive Reasoning Models

Global: Curriculum-Guided Adaptive Recursion Cuts Training Time for Recursive Reasoning Models

Researchers behind a recent arXiv preprint have introduced a training framework called Curriculum-Guided Adaptive Recursion (CGAR) that shortens the training duration of recursive reasoning models on the Sudoku‑Extreme benchmark while maintaining comparable accuracy. The approach delivers a 1.71× speedup—reducing training from 10.93 hours to 6.38 hours—with only a 0.63 percentage‑point drop in final performance.

Background and Motivation

Recursive reasoning models achieve strong problem‑solving abilities by iteratively refining intermediate representations, allowing relatively small networks to rival larger language models. However, the iterative nature of these systems incurs high computational costs; prior work reports up to 36 GPU‑hours for a single Sudoku‑Extreme training run. Existing implementations typically employ a fixed recursion depth and uniform supervision weighting, which can lead to inefficient use of resources.

Curriculum-Guided Adaptive Recursion (CGAR)

CGAR addresses these inefficiencies through two complementary mechanisms. The Progressive Depth Curriculum (PDC) schedules the recursion depth, beginning with shallow configurations (2, 1) and gradually transitioning to deeper settings (6, 3) over three stages, yielding a 41.4% reduction in floating‑point operations. Hierarchical Supervision Weighting (HSW) assigns exponentially decaying importance to supervision steps, which lowers gradient variance by roughly 40% and accelerates convergence.

Performance Gains on Sudoku‑Extreme

When evaluated on the Sudoku‑Extreme task, the full CGAR pipeline achieved a 1.71× training speedup with a marginal 0.63% accuracy decline (from 86.65% to 86.02%). Isolating PDC produced a 2.26× speedup at 85.47% accuracy, representing a Pareto improvement in efficiency versus quality. HSW alone contributed a 1.61× speedup, underscoring the benefit of adaptive supervision.

Inference Efficiency Improvements

Beyond training, models trained with CGAR demonstrated superior inference characteristics, attaining 100% halting accuracy and requiring 11% fewer reasoning steps on average. These gains translate to faster, more predictable deployment in downstream applications that rely on recursive reasoning.

Broader Implications

The ability to treat recursion depth as a scheduled parameter makes recursive models more practical for neurosymbolic AI and program synthesis tasks, especially on modest hardware configurations. By curbing overfitting and reducing computational demand, CGAR could broaden access to advanced reasoning systems across research and industry settings.

Availability and Next Steps

The authors have released the CGAR implementation on GitHub and provided model checkpoints via Hugging Face, enabling reproducibility and further experimentation. Future work may explore extending the curriculum framework to other recursive architectures and problem domains.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.