LLM training in simple, raw C/CUDA

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

Interesting resources related to XAI (Explainable Artificial Intelligence)

Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 30+ benchmarks

🛰️ Official repository of paper "RemoteCLIP: A Vision Language Foundation Model for Remote Sensing" (IEEE TGRS)

Mind

LAVIS - A One-stop Library for Language-Vision Intelligence

Compose multimodal datasets 🎹

Collection of AWESOME vision-language models for vision tasks

🔥 A tool for visualizing and tracking your machine learning experiments. This repo contains the CLI and Python API.

Training Sparse Autoencoders on Language Models

A library for mechanistic interpretability of GPT-style language models

Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

An LLM-powered knowledge curation system that researches a topic and generates a full-length report with citations.

LightEval is a lightweight LLM evaluation suite that Hugging Face has been using internally with the recently released LLM data processing library datatrove and LLM training library nanotron.

Freeing data processing from scripting madness by providing a set of platform-agnostic customizable pipeline processing blocks.

Problem statements, discussions and prototypes

Structured Text Generation

Run any open-source LLMs, such as Llama 3.1, Gemma, as OpenAI compatible API endpoint in the cloud.

PromptBERT: Improving BERT Sentence Embeddings with Prompts

A self-organizing file system with llama 3

Finetune Llama 3.1, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory

Development repository for the Triton language and compiler

Concept Modeling: Topic Modeling on Images and Text

Perplexica is an AI-powered search engine. It is an Open source alternative to Perplexity AI

X bootstrap 1000+ tools and your scripts.

LightXML: Transformer with dynamic negative sampling for High-Performance Extreme Multi-label Text Classification

Benchmarking Benchmark Leakage in Large Language Models

Lists and configuration for our DNS blocking service