Enabling easy statistical significance testing for deep neural networks.

scrape subtitles from opensubtitles.org

Project of llm evaluation to Japanese tasks

Framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.

A one-stop, open-source, high-quality data extraction tool, supports PDF/webpage/e-book extraction.一站式开源高质量数据提取工具，支持PDF/网页/多格式电子书提取。

Thin wrapper for "pandoc" (MIT)

Universal markup converter

A simple toolkit for benchmarking LLMs on mathematical reasoning tasks. 🧮✨

Search for words, documents, images, videos, news, maps and text translation using the DuckDuckGo.com search engine. Downloading files and images to a local hard drive.

Multicultural Proverbs and Sayings

LLM101n: Let's build a Storyteller

Official Codebase for "METAL: Towards Multilingual Meta-Evaluation" (published in Findings of ACL: NAACL 2024)

Convert arctic_shift Reddit data dumps into thread-view documents

Making Reddit data accessible to researchers, moderators and everyone else. Interact with the data through large dumps, an API or web interface.

A Framework for the Systematic Evaluation of Chat-Optimized Language Models as Conversational Agents and an Extensible Benchmark

Code and Data for "PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale" (EMNLP 2023)

NTREX -- News Test References for MT Evaluation

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

Implementations of online merging optimizers proposed by Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment

A native PyTorch Library for large model training

The official repository of the paper "(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts"

[ICLR 2024] COLLIE: Systematic Construction of Constrained Text Generation Tasks

日本語マルチタスク言語理解ベンチマーク Japanese Massive Multitask Language Understanding Benchmark

Multilingual Corpus of Web Fiction

Official code and data of "3AM: An Ambiguity-Aware Multi-Modal Machine Translation Dataset"

NAACL 2024: SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning