Researchers Map Growing Landscape of Threats to Foundation AI Models

Global: Researchers Map Growing Landscape of Threats to Foundation AI Models

Researchers affiliated with multiple institutions released a new pre‑print on arXiv on December 2025 that outlines a comprehensive assessment of security risks targeting foundation models in finance, healthcare and critical infrastructure. The study, titled “Characterizing Machine‑Learning Security Risks Across the Model Lifecycle,” aims to fill gaps in traditional cybersecurity approaches that often overlook machine‑learning‑specific threat vectors.

Methodology and Data Collection

The authors compiled a catalog of 93 distinct threats by drawing from three primary sources: 26 entries from the MITRE ATLAS framework, 12 incidents documented in the AI Incident Database, and 55 additional threats identified through a systematic literature review. To gauge real‑world exposure, the team analyzed 854 GitHub repositories containing Python code related to machine‑learning pipelines and employed a multi‑agent Retrieval‑Augmented Generation (RAG) system—leveraging ChatGPT‑4o with a temperature of 0.4—to mine more than 300 scholarly articles and technical reports. This process produced an ontology‑driven threat graph linking tactics, techniques, procedures (TTPs), vulnerabilities, and lifecycle stages.

Newly Identified Threat Vectors

Among the findings are several previously unreported attack surfaces. The paper highlights commercial large‑language‑model (LLM) API model‑stealing techniques, instances where parameter memorization leads to data leakage, and preference‑guided text‑only jailbreaks that bypass safety constraints without requiring code execution.

Prevalent Tactics, Techniques, and Procedures

Dominant TTPs identified include MASTERKEY‑style jailbreaks that exploit model introspection, federated poisoning attacks that corrupt distributed training data, diffusion backdoors embedded in generative models, and leakage of preference‑optimization parameters. These tactics primarily affect the pre‑training and inference phases of the model lifecycle, where the most critical assets reside.

Supply‑Chain Vulnerabilities in Open‑Source Libraries

Graph analysis of the examined repositories revealed dense clusters of vulnerabilities in libraries that suffer from poor patch propagation. The authors note that many projects lag in updating dependencies, creating a fertile environment for supply‑chain exploits that can cascade into downstream applications.

Proposed Security Measures

To mitigate these risks, the study recommends the development of adaptive, machine‑learning‑specific security frameworks. Key components include rigorous dependency hygiene, continuous threat‑intelligence integration, and real‑time monitoring of model behavior throughout the development and deployment pipeline.

Implications for Stakeholders

The authors conclude that without dedicated safeguards, foundation models remain susceptible to a widening array of attacks that could compromise sensitive sectors. They call on industry, academia, and policy makers to collaborate on establishing standards that address both supply‑chain and inference‑stage vulnerabilities.

This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.