New CAOS Framework Enhances Uncertainty Quantification for One-Shot Machine Learning Models

Global: New CAOS Framework Enhances Uncertainty Quantification for One-Shot Machine Learning Models

A novel conformal inference framework named CAOS—Conformal Aggregation of One-Shot Predictors—was introduced in a paper submitted on Jan 8 2026 and revised on Jan 30 2026. Authored by Maja Waldron, the work proposes a method to provide principled uncertainty estimates for one‑shot prediction tasks while fully exploiting limited labeled data.

Background

One‑shot prediction allows pretrained foundation models to adapt to new tasks using a single labeled example, but it traditionally lacks reliable measures of predictive uncertainty. Existing split‑conformal approaches require data splitting and rely on a single predictor, which can be inefficient when labeled samples are scarce.

Method Overview

CAOS addresses these limitations by adaptively aggregating multiple one‑shot predictors and employing a leave‑one‑out calibration scheme. This design eliminates the need for a separate holdout set, thereby making maximal use of the available labeled instance.

Theoretical Guarantees

Although the approach departs from classical exchangeability assumptions, the authors demonstrate that CAOS achieves valid marginal coverage through a monotonicity‑based argument, ensuring finite‑sample guarantees despite the unconventional data usage.

Experimental Results

Empirical tests on one‑shot facial landmarking and RAFT text classification tasks show that CAOS produces substantially smaller prediction sets than standard split‑conformal baselines while preserving the promised coverage levels.

Potential Impact

If adopted broadly, the framework could improve rapid model adaptation across domains where labeled data are limited, offering both efficiency and trustworthy uncertainty quantification.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.