Welcome to the official GitHub repository for LLM360's website, a collaborative initiative between Petuum, MBZUAI, and Cerebras. This project is dedicated to open-sourcing Large Language Models (LLMs) to promote transparency, trust, and collaborative research in the AI community.

LLM360 aims to redefine open-source in machine learning by providing unprecedented access to the entire training process of LLMs. We release all the intermediate checkpoints, training data, source code, logs, and metrics associated with each model. Our first releases under this initiative are the Amber-7B and CrystalCoder-7B models.

• Details: Trained on 1.2 trillion tokens, Amber-7B offers insights into the learning dynamics of LLMs.
• Resources:
  • Model
  • Metrics
  • Data
  • Code

• Details: A unique model combining text and coding capabilities, trained on 1.4 trillion tokens.
• Resources:
  • Model
  • Metrics
  • Data
  • Code

We encourage community involvement and contributions. Here are ways you can engage with LLM360:

• GitHub: Explore and contribute to our GitHub repository.
• HuggingFace: Access and download our models on HuggingFace.
• Research Contributions: Share your research findings using our models.
• Feedback and Suggestions: Provide your valuable input using our Feedback Form.

This project would not be possible without the support of Petuum, MBZUAI, and Cerebras. We also extend our gratitude to the broader AI research community for their contributions and feedback.

For inquiries or collaborations, feel free to reach out through our Contact Page.

LLM360 models and resources are open-sourced under the Apache 2.0 license.

   Copyright 2023 LLM360

   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at

       http:https://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.