Prompt injection and Data Exfiltration Attacks in Large Language Model Applications: Detection and Mitigation Framework

Abstract

Large Language Models (LLMs) such as GPT-4, Gemini, Claude, and open-source transformer systems are rapidly being embedded into real-world applications including chatbots, enterprise knowledge assistants, healthcare systems, and developer tools. While these models offer unprecedented capabilities in natural language reasoning, they introduce a new class of security vulnerabilities known as prompt injection and data exfiltration attacks. Unlike traditional software exploits that target code or network layers, these attacks manipulate the instruction-following behavior of LLMs through carefully crafted textual inputs. Recent studies demonstrate that malicious prompts can override system rules, expose hidden prompts, and retrieve sensitive data from connected tools and documents. This paper presents an in-depth study of prompt injection mechanisms, analyzes data leakage paths in LLM-powered systems, and proposes a structured detection and mitigation framework for secure LLM deployment. Experimental simulations validate the effectiveness of the framework in reducing adversarial success rates. The findings emphasize the need for AI-aware security practices in modern software architectures.