New Autoregressive Framework Enables Real-Time Streaming Interactive Avatars

Global: New Autoregressive Framework Enables Real-Time Streaming Interactive Avatars

Researchers have unveiled a two‑stage autoregressive adaptation and acceleration framework that transforms a high‑fidelity human video diffusion model into a system capable of real‑time, interactive avatar streaming. The approach combines autoregressive distillation with adversarial refinement to meet the low‑latency demands of digital human applications.

Background and Motivation

Diffusion‑based avatar generators have demonstrated impressive visual quality, yet their non‑causal architectures and substantial computational overhead render them impractical for continuous streaming. Existing interactive solutions often restrict output to the head‑and‑shoulder region, limiting expressive gestures and full‑body motion.

Proposed Framework

The new framework operates in two stages. First, an autoregressive distillation process converts the original diffusion model into a causal, lower‑latency version. Second, an adversarial refinement step restores visual fidelity, ensuring the streamed output remains high‑quality while meeting real‑time constraints.

Key Architectural Components

Three novel elements support long‑term stability and consistency: a Reference Sink that anchors temporal information, a Reference‑Anchored Positional Re‑encoding (RAPR) strategy that aligns spatial features across frames, and a Consistency‑Aware Discriminator that penalizes temporal artifacts during training.

Avatar Model Capabilities

Built on this foundation, the authors present a one‑shot, interactive human avatar capable of generating natural talking and listening behaviors accompanied by coherent gestures. The model accepts a single reference input and produces continuous, full‑body motion without requiring separate pose or gesture modules.

Evaluation and Results

Extensive experiments reported in the preprint indicate state‑of‑the‑art performance, surpassing prior methods in generation quality, real‑time efficiency, and interaction naturalness. Quantitative metrics and user studies both demonstrate measurable improvements over baseline diffusion and autoregressive systems.

Implications and Future Directions

The framework promises to advance digital human research, with potential applications in virtual reality, telepresence, and online education where low‑latency, expressive avatars are essential. Ongoing work aims to further reduce hardware requirements and explore multi‑user interaction scenarios.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.