The paper: http://arxiv.org/abs/2311.11538
## Purpose
This study investigates the vulnerability of over 200 custom [[Generative Pre-trained Transformers (GPTs)]] to prompt injection attacks, highlighting significant security risks in these user-customized AI models.
## Methods
- Crafting adversarial prompts
- Testing over 200 custom GPT models
- Analyzing for system prompt extraction and file leakage
- Conducting red-teaming evaluations against popular prompt injection defenses
## Key Findings
1. Most custom GPT models are susceptible to prompt injection, with a 97.2% success rate for system prompt extraction and a 100% success rate for file leakage.
2. The presence of a code interpreter in GPTs increases the ease of prompt injection.
3. Defensive prompts are not robust enough to prevent system prompt extraction and file leakage.
4. Disabling code interpreters in custom GPTs enhances security but is not a complete solution.
## Discussion
These findings emphasize the urgent need for more robust security frameworks in custom GPT models. They raise awareness about the trade-offs between customizability and security in AI systems.
## Critiques
1. Limited exploration of the impact of diverse GPT configurations on prompt injection vulnerability.
2. The potential overemphasis on technical solutions without addressing broader ethical and policy implications.
3. Reliance on adversarial testing, which may not cover all possible real-world attack scenarios.
## Tags
#AIsecurity #PromptInjection #GPT #CustomGPT #Cybersecurity.