nipundev
Follow
Stars
Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and mo…
Whistleblower is a tool for leaking system prompts and capability discovery of any API accessible LLM App. Built for developers, security red-teams and folks who want to know what's going on inside…
⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs
A curation of awesome tools, documents and projects about LLM Security.
PromtFuzz is an automated tool that generates high-quality fuzz drivers for libraries via a fuzz loop constructed on mutating LLMs' prompts.
Make your GenAI Apps Safe & Secure 🚀 Test & harden your system prompt
fabric is an open-source framework for augmenting humans using AI. It provides a modular framework for solving specific problems using a crowdsourced set of AI prompts that can be used anywhere.
🌐 The Internet OS! Free, Open-Source, and Self-Hostable.
A community wiki for all things AI/ML bill of materials (MLBOM, AIBOM) and transparency into AI/ML models.
Potential WG on Artificial Intelligence and Machine Learning (AI/ML)
A simple threat modeling tool to help humans to reduce time-to-value when threat modeling
Code at the speed of thought – Zed is a high-performance, multiplayer code editor from the creators of Atom and Tree-sitter.
🐢 Open-Source Evaluation & Testing for LLMs and ML models
Reading list of more resources to learn about Adversarial Attacks on AI Systems
Devika is an Agentic AI Software Engineer that can understand high-level human instructions, break them down into steps, research relevant information, and write code to achieve the given objective…
a CLI that provides a generic automation layer for assessing the security of ML models
Find and fix 400+ types of hardcoded secrets and 70+ types of infrastructure-as-code misconfigurations.
A LLM explicitly designed for getting hacked
NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Analysis of the Enterprise SAST/DAST product landscape
A curated list of resources related to privacy engineering
The core library of differential privacy algorithms powering the OpenDP Project.