Adversarial Prompts Exploit Load Imbalance in Mixture-of-Experts Language Models

Global: Adversarial Prompts Exploit Load Imbalance in Mixture-of-Experts Language Models

Researchers have identified a denial-of-service vulnerability in large language models that employ Mixture-of-Experts (MoE) architectures. The study, posted to arXiv in December 2025, demonstrates that specially crafted out-of-distribution prompts can force the model’s router to concentrate token assignments on a limited subset of experts. This concentration creates computational bottlenecks on certain devices while leaving others idle, thereby violating expected service-level agreements for response time.

Mixture-of-Experts models achieve parameter efficiency by distributing thousands of expert sub-networks across multiple processing units. During inference, expert parallelism assigns each token to the top‑k experts based on a learned routing function, without an explicit constraint to balance the workload across devices.

Vulnerability Origin

The absence of explicit load-balancing constraints allows adversarial inputs to manipulate the routing strategy. By presenting prompts that lie outside the model’s training distribution, attackers can cause the router to repeatedly select the same experts for every token, creating a severe imbalance in computational demand.

RepetitionCurse Attack

The authors propose a technique called RepetitionCurse, which exploits the identified flaw using a low-cost black-box approach. The method constructs adversarial prompts composed of simple repetitive token patterns that are effective across different MoE models without requiring internal model knowledge.

Empirical Impact

Testing on the publicly available Mixtral-8x7B model, RepetitionCurse increased end-to-end inference latency by a factor of 3.063, substantially degrading service availability. The latency spike directly impacts the time-to-first-token metric, a key performance indicator for interactive applications.

Potential Mitigations

Mitigation strategies may include integrating dynamic load-balancing checks into the routing algorithm, monitoring expert utilization patterns for anomalies, and limiting the influence of out-of-distribution inputs on routing decisions. Providers are advised to evaluate these safeguards as part of routine security assessments.

Broader Implications

The findings underscore the need for comprehensive security testing of AI inference pipelines, especially as MoE architectures become more prevalent in commercial deployments. Addressing routing-related vulnerabilities can help maintain reliability and protect against denial-of-service exploits.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.