ai_ml_hacking
Awesome AI & ML Pentesting: The Curated Resource Every AI Security Researcher Needs
Artificial Intelligence has become one of the fastest-growing attack surfaces in modern cybersecurity. Organizations are deploying Large Language Models (LLMs), Retrieval-Augmented Generation (RAG) systems, autonomous agents, Model Context Protocol (MCP) servers, AI copilots, and machine learning pipelines at an unprecedented pace.
Unfortunately, security knowledge around these technologies is scattered across hundreds of GitHub repositories, research papers, blog posts, OWASP projects, academic publications, and conference talks.
That’s exactly why I created the Awesome AI & ML Pentesting repository.
GitHub Repository: https://github.com/raj77in/awesome-ai-ml-pentesting
The goal is simple:
Build one of the most comprehensive collections of resources for AI security, LLM hacking, adversarial machine learning, AI red teaming, prompt injection, MCP security, and autonomous AI pentesting.
Whether you’re a penetration tester, bug bounty hunter, AI engineer, security researcher, or student, this repository aims to become your go-to reference.
Why AI Pentesting Is Different
Traditional penetration testing focuses on applications, networks, APIs, cloud infrastructure, and operating systems.
AI introduces entirely new attack surfaces:
- Prompt Injection
- Indirect Prompt Injection
- Jailbreaking
- Context Poisoning
- Model Theft
- Training Data Poisoning
- Embedding Manipulation
- Vector Database Attacks
- RAG Exploitation
- Tool Abuse
- MCP Security
- Autonomous Agent Exploitation
- Model Evasion
- Token Manipulation
- Hallucination Abuse
- Sensitive Information Disclosure
Most existing pentesting methodologies simply don’t cover these problems.
As AI adoption accelerates, security professionals need an entirely new knowledge base.
Why This Repository Exists
When I started researching AI security, I quickly realized that useful information was spread everywhere:
- one GitHub repo for prompt injection
- another for adversarial ML
- a separate OWASP project
- academic papers on arXiv
- conference talks
- random blog posts
- dozens of independent AI hacking tools
Finding high-quality material required hours of searching.
This repository brings those resources together into one curated collection.
Instead of searching for hours, you can simply browse categories and immediately find the best references.
What You’ll Find
The repository covers virtually every major area of AI security.
LLM Security
Learn how attackers interact with language models:
- Prompt Injection
- Indirect Prompt Injection
- Jailbreaking
- System Prompt Leakage
- Context Window Attacks
- Role Manipulation
- Tool Calling Abuse
- Prompt Stealing
Machine Learning Security
Classic machine learning remains relevant.
Topics include:
- Adversarial Examples
- Data Poisoning
- Model Poisoning
- Model Extraction
- Membership Inference
- Privacy Attacks
- Evasion Attacks
- Backdoor Models
AI Red Teaming
The repository includes numerous resources covering structured AI security assessments.
Topics include:
- AI threat modeling
- red team methodologies
- evaluation frameworks
- automated testing
- benchmark suites
- security testing checklists
RAG Security
Retrieval-Augmented Generation has become one of the largest enterprise AI attack surfaces.
Resources cover:
- vector databases
- embedding attacks
- document poisoning
- retrieval manipulation
- malicious documents
- context injection
- chunk poisoning
MCP Security
Model Context Protocol is rapidly becoming the standard interface for AI agents.
Like every new protocol, it introduces unique risks.
The repository contains resources covering:
- MCP architecture
- tool security
- trust boundaries
- authentication
- authorization
- tool isolation
- prompt injection across tools
- agent orchestration risks
Autonomous AI Agents
Agentic AI is changing penetration testing.
Modern AI agents can:
- plan
- execute commands
- use tools
- browse documentation
- analyze responses
- iterate toward objectives
This creates both opportunities and entirely new attack vectors.
The repository includes frameworks, research, and practical implementations covering autonomous AI security and offensive workflows. Recent open-source projects are increasingly combining LLMs with MCP servers and traditional offensive security tooling to automate portions of reconnaissance, testing, and reporting.
Prompt Engineering for Security
Effective AI security testing isn’t only about attacking models.
Understanding prompt engineering helps you:
- create better security assistants
- improve AI workflows
- reduce hallucinations
- design safer agents
- build robust system prompts
Research Papers
One of the strongest sections of the repository is the academic paper collection.
Instead of manually searching arXiv every week, you’ll find curated papers covering:
- LLM security
- adversarial ML
- autonomous pentesting
- AI agents
- defensive AI
- attack automation
- model robustness
Research over the past few years has shown both the promise and limitations of LLM-assisted penetration testing, from planning and vulnerability analysis to automated action loops.
Tools Included
The repository references numerous open-source projects from the AI security ecosystem.
Examples include:
- AI security scanners
- LLM testing tools
- prompt injection frameworks
- adversarial ML libraries
- red teaming utilities
- autonomous AI agents
- MCP servers
- evaluation frameworks
- security benchmarks
Rather than endorsing any single project, the repository acts as a curated directory that helps practitioners discover and compare available options.
Who Should Use This Repository?
Penetration Testers
Learn how AI applications differ from traditional web applications.
Bug Bounty Hunters
Understand where AI vulnerabilities exist before everyone else.
AI Engineers
Build secure LLM applications from the beginning.
Security Researchers
Find papers, tools, benchmarks, and references in one place.
Students
Instead of wondering where to begin, simply work through the repository category by category.
Learning Path
If you’re new to AI security, here’s a practical roadmap:
- Learn how LLMs work.
- Understand prompt engineering.
- Study the OWASP LLM Top 10.
- Explore prompt injection techniques.
- Learn RAG architecture.
- Study vector databases.
- Understand AI agents.
- Learn MCP.
- Explore adversarial machine learning.
- Practice with intentionally vulnerable AI applications.
This progression builds a strong foundation before moving into advanced attack techniques.
The Future of Penetration Testing
AI isn’t replacing penetration testers.
It’s changing how they work.
Tomorrow’s security professionals will increasingly use AI to:
- automate reconnaissance
- summarize findings
- generate payloads
- analyze source code
- reason about attack paths
- review logs
- assist in exploit development
- document engagements
At the same time, defenders must understand how to evaluate the security of AI systems themselves. Community discussions consistently point toward a human-in-the-loop model, where AI accelerates repetitive tasks while experienced testers validate findings and investigate complex attack chains.
Contributing
AI security evolves almost daily.
New research papers, attack techniques, frameworks, benchmarks, and tools appear every week.
If you discover something valuable that’s missing from the repository, contributions are welcome.
The goal isn’t simply to create another “awesome list.”
The goal is to build one of the most useful community resources for AI security practitioners.
Final Thoughts
The cybersecurity industry has spent decades developing mature methodologies for web applications, networks, cloud infrastructure, and operating systems.
Artificial Intelligence represents the next major frontier.
Whether your interest is LLM security, adversarial machine learning, AI red teaming, RAG exploitation, or autonomous agents, having a single, organized collection of high-quality resources can dramatically reduce the time spent searching and increase the time spent learning and testing.
If you’re working in offensive security—or planning to—bookmark the repository, explore the resources, and contribute back when you find something valuable.
The AI security ecosystem is evolving rapidly, and staying current has never been more important.
Repository: https://github.com/raj77in/awesome-ai-ml-pentesting