New Framework Evaluates Neural Network Similarity Across Architectures and Pruning

Global: New Framework Evaluates Neural Network Similarity Across Architectures and Pruning

Introducing the Triangle of Similarity

Researchers have presented a novel analytical framework called the Triangle of Similarity to assess how neural network representations compare across different models. The work, posted to arXiv in January 2026, aims to give scientists a more comprehensive view of internal model mechanisms for validation and selection.

Three Complementary Perspectives

The framework integrates three distinct similarity measures: static representational similarity using techniques such as CKA and Procrustes analysis; functional similarity captured through Linear Mode Connectivity or Predictive Similarity; and sparsity similarity that evaluates robustness under pruning.

Experimental Scope

To test the approach, the authors examined a broad set of convolutional neural networks, Vision Transformers, and Vision‑Language models. Both in‑distribution data (ImageNetV2) and out‑of‑distribution data (CIFAR‑10) served as evaluation testbeds, providing insight into how models behave under varied conditions.

Architectural Influence on Similarity

Initial results indicate that the architectural family of a model is the dominant factor shaping representational similarity, with models clustering according to their design lineage rather than training specifics.

Pruning Dynamics and Accuracy

The study finds a strong correlation between CKA self‑similarity and task accuracy during pruning phases, although accuracy tends to decline more sharply than similarity metrics suggest.

Regularization Effects of Pruning

For certain model pairs, pruning appears to act as a regularizer, simplifying representations and exposing a shared computational core that persists despite weight reduction.

Potential Applications

By offering a multi‑angle assessment, the Triangle of Similarity could aid researchers in selecting models that not only perform well but also converge on comparable internal processes, supporting reproducibility in scientific applications.

Next Steps

Future investigations may expand the framework to additional model families and explore how the identified similarity clusters relate to downstream task generalization.
This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.