New Zeroth-Order Framework Enhances Privacy and Efficiency in Vertical Federated Learning

Global: New Zeroth-Order Framework Enhances Privacy and Efficiency in Vertical Federated Learning

Breakthrough in Private Model Training

Researchers have introduced DPZV, a communication‑efficient, differentially private zeroth‑order vertical federated learning (VFL) framework that aims to curb both the bandwidth demands and privacy vulnerabilities inherent in traditional VFL systems. The approach injects calibrated scalar‑valued differential privacy noise into the server‑to‑device downlink, offering tunable privacy guarantees while preserving gradient quality.

Challenges in Existing VFL Architectures

Vertical federated learning enables collaborative model development across devices that hold complementary feature sets, but the exchange of loss‑related gradients from server to participants can reveal sensitive information. Prior privacy‑preserving methods typically add differential privacy noise directly to gradient vectors, which often inflates variance, slows convergence, and necessitates additional communication rounds.

DPZV’s Zeroth‑Order Optimization Strategy

DPZV leverages zeroth‑order (ZO) optimization, estimating gradients through function evaluations rather than explicit gradient transmission. By applying scalar DP noise to these ZO estimates, the framework reduces variance amplification compared with vector‑level noise injection, thereby maintaining more accurate learning signals.

Theoretical Foundations

The authors provide rigorous proofs that DPZV satisfies $(epsilon, delta)$‑differential privacy and attains convergence rates comparable to first‑order DP‑SGD, despite relying solely on ZO estimators. These guarantees hold under standard smoothness and bounded‑variance assumptions.

Empirical Validation

Extensive experiments on benchmark datasets demonstrate that DPZV consistently outperforms existing DP‑VFL baselines in privacy‑utility trade‑offs. Under strict privacy budgets ((epsilon le 10)), the method requires fewer communication rounds while achieving comparable or higher predictive accuracy.

Implications and Future Directions

The results suggest that scalar‑noise ZO techniques can reconcile the competing demands of privacy, communication efficiency, and model performance in VFL settings. Ongoing work aims to extend DPZV to heterogeneous model architectures and to evaluate its robustness against advanced inference attacks.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.