As artificial intelligence (AI) progresses and embeds itself into every facet of digital life, its applications in cyberdefense become increasingly critical. This repository serves as a comprehensive compilation of resources pertaining to AI-based cyberdefense. It curates a wide spectrum of materials, ranging from books and articles on general cybersecurity and AI cybersafety, to the application of machine learning techniques in cybersecurity. Additionally, it provides an overview of prevalent cyber attacks and potential defenses, highlights relevant conferences and events, and introduces products and initiatives from leading AI organizations committed to strengthening cyber defense. By equipping readers with this knowledge, we aim to empower individuals, organizations, and nation-states to leverage AI technologies in fortifying their cyber infrastructure and effectively combat the rising tide of cyber threats. This work underscores the urgent need for informed and proactive engagement in this rapidly evolving landscape of AI and cyber defense.

• AI Cyberdefense
  • Resources
    • General cybersecurity
    • AI cybersafety
    • Other lists
  • Experimental resources
    • Datasets
    • Packages
    • References
    • Project ideas
  • Conferences and events
  • Products
  • AI organization commitments
  • Notes
    • Overview of attacks and defenses

• [list] Goodreads list of books
  • [book] Silence on the Wire: It is supposedly one of the better introductions to "how the internet works"
• [post] Overview in Danish
• [report] MIT Cybersecurity review of 20 countries
• [website] Live cyber threat map
• [whitepaper] Checkpoint's guide for adopting a threat prevention approach to cybersecurity
• [report] IBM's estimates for costs of data breaches
• [article] Building a vulnerability benchmark
• [guidelines] NIST Framework for Improving Critical Infrastructure Cybersecurity
• [article] Social cybersecurity: an emerging science
• [textbook] Cybersecurity for Industry 4.0

• [article] Review: machine learning techniques applied to cybersecurity
• [article] Cybersecurity data science: an overview from machine learning perspective
• [article] Machine learning approaches to IoT security: A systematic literature review
• [sequence] AI infosec: first strikes, zero-day markets, hardware supply chains, adoption barriers
• [post] AI Safety in a World of Vulnerable Machine Learning Systems

• BlueTeam-Tools: This github repository contains a collection of 65+ tools and resources that can be useful for blue teaming activities.
• RedTeam-Tools: This github repository contains a collection of 130+ tools and resources that can be useful for red teaming activities.
• awesome-security
• Reseach-AI-CyberSecurity: A collection of resources to start off researching AI in CyberSecurity
• Awesome Cyber Security: A collection of awesome software, libraries, documents, books, resources and cool stuff about security.
• Awesome AI for cybersecurity: Awesome list of AI for cybersecurity including network (network traffic analysis and intrusion detection), endpoint (anti-malware), application (WAF or database firewalls), user (UBA), process behavior (anti-fraud).
• Awesome Machine Learning for Cyber Security: A curated list of amazingly awesome tools and resources related to the use of machine learning for cyber security.
• Awesome AI Security: A curated list of AI security resources inspired by awesome-adversarial-machine-learning & awesome-ml-for-cybersecurity.

We're interested in running experiments on how we can make cybersecurity safer or increase the reliability and defense of LLM systems.

• A really good overview of real-world networking datasets and resources
• Darknet dataset (∆)

• [python] FlowLabeler: Processing IP packets
• [python] Malware environment for OpenAI Gym: Create an AI that learns through reinforcement learning which functionality-preserving transformations to make on a malware sample to break through / bypass machine learning static-analysis malware detection.

• EvadeRL: Evading PDF Malware Classifiers with Deep Reinforcement Learning

• Malware detection: AI has the potential to provide much more accurate and faster detection of malicious activity than traditional signature-based detection. To design this kind of system, you would need to first create a data set of network traffic. This data set would need to include both malicious and benign traffic so that the AI could learn to distinguish between the two. | Network traffic dataset
• LLM Phishing Detection: Train an LLM to generate phishing emails and use it as a benchmark to train and test anti-phishing systems.
• Input Sanitization Check: Test various unsanitized inputs to an LLM and observe if it can be exploited to perform unintended operations, such as SQL injection or Cross-Site Scripting (XSS). (dataset 1, dataset 2)
• Malicious Code Generation Prevention: Test different safety mitigations in preventing an LLM from generating harmful code snippets, even when specifically requested. This can involve testing various prompts and fine-tuning strategies.
• Safety Layers Benchmarking: Evaluate the effectiveness of various safety layers (rate limiters, use-case specific guidelines) in protecting the LLM from misuse.
• LLM Chatbot Resilience: Evaluate how well an LLM chatbot can withstand attempted attacks or malicious uses by simulating an adversarial user trying to trick the system into generating harmful content.
• Evaluating LLMs for Intrusion Detection: Test LLMs' capability to detect intrusion attempts in network traffic data, compared to traditional IDS systems.
• Exploit Generation Prevention: Evaluate the ability of LLMs to generate known software exploits when given a description of a vulnerability. The aim is to prevent the model from generating such exploits.
• Content Filtering Effectiveness: Evaluate the effectiveness of content filtering mechanisms in LLMs in blocking the generation of malicious content.
• LLM Robustness to Adversarial Attacks: Test the robustness of an LLM to adversarial attacks, where inputs are deliberately crafted to mislead the model or cause it to generate malicious outputs.
• Differential Privacy Implementation: Implement differential privacy techniques to protect sensitive information in LLM training data and evaluate how this affects the model's ability to generate malicious content.

• Palantir AIP
• CIS Benchmarks
• Intel Owl: Single API to get threat information about any file, IP, etc.

• Anthropic Trust
• OpenAI Trust