Summarize existing representative LLMs text datasets.

Official implementation for the paper *🎯DART-Math: Difficulty-Aware Rejection Tuning for Mathematical Problem-Solving*

A collection of guides and examples for the Gemma open models from Google.

Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"

A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"

Temporal question answering dataset for Wikidata

Dialogue Action Tokens: Steering Language Models in Goal-Directed Dialogue with a Multi-Turn Planner

Q-Probe: A Lightweight Approach to Reward Maximization for Language Models

Official implementation of DPFM @ ICLR 2024 paper "Augmenting Math Word Problems via Iterative Question Composing"(https://arxiv.org/abs/2401.09003)

APPS: Automated Programming Progress Standard (NeurIPS 2021)

A (deprecated) framework for building exercises to work with Khan Academy.

Code and data for "MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models"

[ACL 2024] MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi-Turn Dialogues

A framework for serving and evaluating LLM routers - save LLM costs without compromising quality!

Leveraging BERT and c-TF-IDF to create easily interpretable topics.

The opensoure repository of FuzzLLM

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Arena-Hard-Auto: An automatic LLM benchmark.

[ICLR 2024] SWE-Bench: Can Language Models Resolve Real-world Github Issues?

Scalable toolkit for efficient model alignment

LiveBench: A Challenging, Contamination-Free LLM Benchmark

tiktoken is a fast BPE tokeniser for use with OpenAI's models.

A platform for developing AI systems as described in A Roadmap towards Machine Intelligence - http:https://arxiv.org/abs/1511.08130

Simple language-driven navigation tasks for studying compositional learning

Specify what you want it to build, the AI asks for clarification, and then builds it.

🐚 OpenDevin: Code Less, Make More