Researchers Propose Post-Training Ternarization Framework for Large Language Models

Global: Researchers Propose Post-Training Ternarization Framework for Large Language Models

A team of researchers has introduced PT⁲‑LLM, a post‑training ternarization framework designed to compress large language models (LLMs) while preserving performance. The work, posted on arXiv in October 2025, aims to lower memory consumption and accelerate inference by converting full‑precision weights to ternary values.

Motivation for Model Compression

LLMs have demonstrated strong capabilities across a range of tasks, yet their substantial memory and compute requirements limit deployment on edge devices and cost‑sensitive environments. Compression techniques such as quantization are therefore critical for broader accessibility.

Challenges in Post‑Training Quantization

Applying ternarization after training presents two primary obstacles: the absence of gradient‑based fine‑tuning and the presence of outlier weights that can cause large quantization errors. Existing post‑training quantization (PTQ) methods often struggle to balance accuracy with the aggressive reduction to three‑level representations.

PT⁲‑LLM Framework Overview

The proposed PT⁲‑LLM framework centers on an Asymmetric Ternary Quantizer equipped with a two‑stage refinement pipeline. First, Iterative Ternary Fitting (ITF) alternates between constructing an optimal ternary grid and applying flexible rounding to minimize quantization error. Second, Activation‑aware Grid Alignment (AGA) further adjusts the grid to better align with full‑precision activation outputs.

Structural Similarity‑Based Reordering

To mitigate the impact of outliers, the authors add a plug‑and‑play Structural Similarity‑based Reordering (SSR) strategy. SSR leverages inter‑column structural similarity within weight matrices, reordering columns to create a more uniform distribution that eases ternarization.

Experimental Validation

Extensive experiments on benchmark LLMs show that PT⁲‑LLM achieves competitive accuracy compared with state‑of‑the‑art 2‑bit PTQ methods, while using less memory. The framework also delivers end‑to‑end speedups during both prefill and decoding phases, indicating practical inference benefits.

Implications and Future Directions

By demonstrating that ternarization can be effectively applied in a post‑training setting, the study opens avenues for deploying large models on resource‑constrained hardware without extensive retraining. The authors plan to release code and model checkpoints to facilitate further research.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.