New Prompting Framework Boosts Phishing URL Detection with Large Language Models

Global: New Prompting Framework Boosts Phishing URL Detection with Large Language Models

Background

Researchers at the University of Sydney have introduced a novel prompting strategy called Least-to-Most to improve the classification of phishing URLs using large language models (LLMs). The study, posted on arXiv on January 26, 2026, addresses the persistent challenge of accurately identifying malicious web links, a critical concern for both users and cybersecurity professionals.

Methodology

The proposed framework incorporates an “answer sensitivity” mechanism that guides the iterative prompting process, allowing the model to refine its reasoning step by step. By starting with the simplest sub‑task and progressively tackling more complex components, the approach seeks to enhance the model’s logical deductions without extensive supervised training.

Evaluation Results

Experiments were conducted on three publicly available URL datasets and involved four state‑of‑the‑art LLMs. Compared with a one‑shot prompting baseline, the Least-to-Most method achieved higher prediction accuracy and delivered performance comparable to a fully supervised model, despite requiring significantly less labeled data.

Analysis of Iterative Reasoning

The authors’ analysis attributes the gains to the iterative reasoning enabled by the framework. The answer sensitivity component appears to prioritize informative intermediate outputs, which in turn steer subsequent prompts toward more accurate conclusions.

Implications for Cybersecurity

These findings suggest that sophisticated prompting techniques can narrow the gap between few‑shot and fully supervised detection systems. Organizations seeking rapid deployment of phishing defenses may benefit from leveraging LLMs with such prompting strategies, reducing the need for large, continuously updated training corpora.

Future Directions

The research team plans to extend the framework to other security‑related classification tasks and to explore integration with real‑time threat monitoring pipelines. The full experimental setup and code are publicly available in a GitHub repository linked in the paper.This report is based on information from arXiv, licensed under Academic Preprint / Open Access. Based on the abstract of the research paper. Full text available via ArXiv.