🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Inference code for Llama models

A Gradio web UI for Large Language Models.

Making large AI models cheaper, faster and more accessible

TensorFlow code and pre-trained models for BERT

An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Streamlit — A faster way to build and share data apps.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

Finetune Llama 3.2, Mistral, Phi, Qwen 2.5 & Gemma LLMs 2-5x faster with 80% less memory

PyTorch implementations of Generative Adversarial Networks.

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Fast and memory-efficient exact attention

newspaper3k is a news, full-text, and article metadata extraction in Python 3. Advanced docs:

Letta (formerly MemGPT) is a framework for creating LLM services with memory.

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

GLM-130B: An Open Bilingual Pre-Trained Model (ICLR 2023)

An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries

A large-scale 7B pretraining language model developed by BaiChuan-Inc.

Language Technology Platform

汉字转拼音(pypinyin)

Tools for merging pretrained large language models.

📊 A simple command-line utility for querying and monitoring GPU status

中文语言理解测评基准 Chinese Language Understanding Evaluation Benchmark: datasets, baselines, pre-trained models, corpus and leaderboard

A LITE BERT FOR SELF-SUPERVISED LEARNING OF LANGUAGE REPRESENTATIONS, 海量中文预训练ALBERT模型

FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.

[EMNLP 2021] SimCSE: Simple Contrastive Learning of Sentence Embeddings https://arxiv.org/abs/2104.08821

A library for Multilingual Unsupervised or Supervised word Embeddings