New Fractional-Power Neural Network Architecture Improves Approximation of Singular Functions

Global: New Fractional-Power Neural Network Architecture Improves Approximation of Singular Functions

Researchers have introduced M“untz-Sz'asz Networks (MSN), a neural network architecture designed to better approximate functions that exhibit singular or fractional power behavior. The preprint, posted on arXiv, details how the model replaces conventional smooth activation functions with learnable fractional power bases, addressing challenges commonly encountered in physics‑related modeling tasks.

Architecture Overview

Each edge in an MSN computes a composite function of the form (phi(x) = sum_k a_k |x|^{mu_k} + sum_k b_k operatorname{sign}(x) |x|^{lambda_k}), where the exponents (mu_k) and (lambda_k) are trained alongside the coefficients (a_k) and (b_k). This formulation allows the network to adapt its activation profile to the specific singularities present in the target function.

Theoretical Guarantees

The authors prove that MSN inherits universal approximation capabilities from the classical Müntz‑Szász theorem. They also derive approximation rates indicating that, for functions of the form (|x|^{alpha}), MSN can achieve an error of (mathcal{O}(|mu-alpha|^2)) with a single learned exponent, whereas standard multilayer perceptrons would require (mathcal{O}(epsilon^{-1/alpha})) neurons to reach comparable accuracy.

Empirical Performance

Experimental evaluation on supervised regression tasks involving singular target functions shows that MSN reduces prediction error by a factor of 5 to 8 while using roughly one‑tenth the number of parameters of conventional MLPs. These results suggest a substantial efficiency gain in settings where model size and training resources are constrained.

Applications to Physics‑Informed Neural Networks

When applied to physics‑informed neural networks (PINNs), MSN demonstrated improvements of 3 to 6 times on benchmark problems that include singular ordinary differential equations and stiff boundary‑layer equations. The learned exponents were reported to align with the known analytical structure of the solutions, providing an interpretable link between the model and underlying physics.

Implications for Scientific Computing

The study illustrates how incorporating classical approximation theory into neural network design can yield measurable performance benefits for scientifically motivated function classes. The authors suggest that further exploration of theory‑guided architectures may enhance the reliability and efficiency of machine‑learning tools across a range of engineering and physical science applications.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.