New Activation Function Aims to Boost Implicit Neural Representations

Global: New Activation Function Aims to Boost Implicit Neural Representations

Researchers Michal Jan Wlodarczyk, Danzel Serrano and Przemyslaw Musialski submitted a preprint on 10 January 2026 that proposes a novel activation function, called the Hyperbolic Oscillator with Saturation Control (HOSC), for use in implicit neural representations (INRs). The work appears on the arXiv repository and seeks to address gradient instability and limited multi‑scale control that can affect periodic activations such as sine.

Technical Overview

HOSC is defined mathematically as HOSC(x)=tanh(beta sin(ω₀ x)), where β is an explicit scaling parameter and ω₀ sets the base frequency. By embedding the hyperbolic tangent, the activation retains the periodic carrier of sine while introducing a saturation mechanism that bounds the Lipschitz constant at β ω₀.

Gradient Control Mechanism

The inclusion of β gives practitioners a direct lever to tune gradient magnitudes during training. According to the authors, adjusting β allows the activation to remain stable across high‑frequency regimes without sacrificing the ability to represent fine details.

Empirical Evaluation

The authors report a comprehensive empirical study covering image reconstruction, audio synthesis, video rendering, neural radiance fields (NeRFs) and signed distance functions (SDFs). Standardized training protocols were employed, and results are presented for each domain to illustrate the practical impact of the activation.

Comparison with Existing Methods

Benchmarking against established periodic activations such as SIREN and FINER shows that HOSC delivers substantial improvements in certain tasks while achieving parity in others. The paper highlights scenarios where the saturation control yields faster convergence or higher fidelity.

Potential Applications

Given its versatility, HOSC could be integrated into a range of INR‑based pipelines, including 3‑D scene reconstruction, high‑resolution texture synthesis, and audio waveform modeling. The authors provide domain‑specific guidance on selecting β and ω₀ to match the frequency characteristics of the target data.

Future Directions

The authors indicate that further research will explore adaptive schemes for β, as well as extensions to other neural architectures that benefit from periodic behavior. Source code and additional resources are made available through a project page linked in the preprint.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.