Unified Efficient Fine-Tuning of 100+ LLMs (ACL 2024)

记录本人整理的一些数据集

All-in-one text de-duplication

TencentLLMEval is a comprehensive and extensive benchmark for artificial evaluation of large models that includes task trees, standards, data verification methods, and more.

The RedPajama-Data repository contains code for preparing large datasets for training large language models.

Convert WIKI dumped XML (Chinese) to human readable documents in markdown and txt.

A tool for extracting plain text from Wikipedia dumps

This is a repository using the Wiki Extractor to build and prepare WIKIPEDIA for use in tensorflow.

We release a dataset based on Wikipedia sentences and the corresponding translations in 6 different languages along with the scores (scale 1 to 100) generated though human evaluations that represen…

An automatic evaluator for instruction-following language models. Human-validated, high-quality, cheap, and fast.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

活字通用大模型

沉浸式双语网页翻译扩展 , 支持输入框翻译， 鼠标悬停翻译， PDF, Epub, 字幕文件, TXT 文件翻译 - Immersive Dual Web Page Translation Extension

TigerBot: A multi-language multi-task LLM

12306 订票程序，自动登录，自动下单

🦜🔗 Build context-aware reasoning applications

BELLE: Be Everyone's Large Language model Engine（开源中文对话大模型）

Aligning pretrained language models with instruction data generated by themselves.

Personal short implementations of Machine Learning papers

A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

天涯 kkndme 神贴聊房价

Code for "Learning to summarize from human feedback"

Human preference data for "Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback"

ONNX Model Exporter for PaddlePaddle

Bio-Computing Platform Featuring Large-Scale Representation Learning and Multi-Task Deep Learning “螺旋桨”生物计算工具集

Windows Calculator: A simple yet powerful calculator that ships with Windows

Learn Classical Statistical Machine Translation Systems.

GIZA++ is a statistical machine translation toolkit that is used to train IBM Models 1-5 and an HMM word alignment model. This package also contains the source for the mkcls tool which generates th…

TensorFlow code and pre-trained models for BERT