New Interpolating Information Criterion Proposed for Overparameterized Models

Global: New Interpolating Information Criterion Proposed for Overparameterized Models

A team of researchers including Liam Hodgkinson, Chris van der Heide, Robert Salomone, Fred Roosta, and Michael W. Mahoney announced a new information‑theoretic metric designed for model selection in overparameterized machine learning settings. The work, initially submitted to arXiv on 15 July 2023 and revised on 10 January 2026, aims to address limitations of classical criteria when the number of parameters exceeds the data size.

Background on Overparameterization

Traditional information criteria such as AIC or BIC assume a large‑sample regime and penalize model complexity, which can be inappropriate for modern deep‑learning models that often interpolate training data while still generalizing well. Researchers note that this mismatch motivates the search for criteria that reflect the statistical behavior of interpolating estimators.

Bayesian Duality Insight

The authors demonstrate that for any overparameterized model there exists a dual underparameterized model sharing the same marginal likelihood. This Bayesian duality enables the application of classical marginal‑likelihood‑based methods to the overparameterized regime, providing a theoretical bridge between the two settings.

Introducing the Interpolating Information Criterion

Leveraging the duality result, the paper defines the Interpolating Information Criterion (IIC). The IIC incorporates the prior distribution explicitly, accounts for geometric and spectral characteristics of the model, and adjusts for prior misspecification. According to the abstract, the criterion aligns with known empirical and theoretical patterns observed in overparameterized models.

Practical Implications

If validated empirically, the IIC could serve as a tool for practitioners to compare competing overparameterized architectures without relying solely on validation‑set performance. By embedding prior information, the metric may also guide the selection of regularization strategies.

Future Directions

The authors suggest that further work will test the IIC across diverse datasets and model families, and explore extensions that integrate additional aspects of model uncertainty. Such investigations could clarify the robustness of the criterion in real‑world applications.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.